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Abstract 

Previous  research  has  resulted  in  the  development  of 
instruments  to  measure  organizational  effectiveness.  This 
thesis  attempts  to  develop  an  instrument  to  measure  how 
effective  the  organization's  metrics  are  in  relation  to 
their  organization  effectiveness  model,  specifically,  to  the 
Competing  Values  Model  of  organization  effectiveness 
developed  by  Quinn  and  Rohrbaugh.  This  should  aid  managers 
in  determining  whether  or  not  they  are  measuring  what  they 
think  they  are  measuring  and  how  effective  they  are  at 
measuring  organizational  effectiveness.  The  instrument 
developed  can  help  determine  the  specific  aspects  of 
organizational  effectiveness  which  metrics  are  measuring 
according  to  the  Competing  Values  Model. 
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DEVELOPMENT  OF  A  METRIC  EVALUATION 
INSTRUMENT  FOR  USE  IN  MEASURING 
ORGANIZATIONAL  EFFECTIVENESS 

I.  Introduction 

Background 

The  Air  Force  has  directed  its  commands  to  implement 
Total  Quality  Management (TQM)  as  a  way  of  doing  business. 

The  push  in  this  new  direction  is  supposed  to  help  the 
entire  Air  Force  stay  focused  on  the  mission  and  provide 
uniformity  of  purpose  to  its  units.  The  Air  Force  commonly 
addresses  TQM  as  Quality  Air  Force  (QAF) .  QAF  is  intended 
to  help  the  Air  Force  streamline  processes  to  accommodate 
the  recent  reductions  in  personnel  and  organizational 
restructuring  throughout  the  Air  Force.  Doing  higher 
quality  work  with  less  resources  is  key  for  the  Air  Force  to 
successfully  achieve  its  mission.  With  the  recent 
implementation  of  QAF  in  the  Air  Force,  there  is  a  need  to 
know  how  well  the  Air  Force  is  achieving  its  QAF  goals.  In 
order  to  do  this  an  effective  system  of  process  and  quality 
measurement  is  essential.  Unsuccessful  companies  tend  to 
focus  exclusively  on  financial  measures  and  often  lack  some 
essential  measures  of  quality.  By  lacking  such  measures, 
they  learn  of  their  quality  problems  only  after  severe 
damage  has  already  been  done  to  their  organization  (Juran, 
1993:36) . 
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since  an  organization  must  measure  processes  to  measure 
organizational  effectiveness,  how  do  they  know  when  they 
have  a  good  measure?  "Recently  the  Air  Force  Quality 
Institute  has  begun  to  question  whether  or  not  the  way  we 
are  measuring  things  and  developing  our  measures  is  actually 
productive"  (Stone,  1995) .  Additionally  there  appears  to  be 
confusion  about  metrics  (measurements  designed  to  identify 
improvement  areas  of  processes) ,  their  design  and  what  they 
are  used  for.  Many  organizations  today  measure  themselves 
to  baseline  or  benchmark  their  performance.  The  approaches 
used  vary  based  on  a  wide  range  of  assumptions  about  what 
constitutes  effectiveness.  Some  organizations  take  a  snap¬ 
shot  of  themselves  by  using  metrics  to  measure  efficiency, 
effectiveness,  and  productivity.  Often  organizations 
measure  what  is  convenient  or  easy  to  measure,  rather  than 
what  is  meaningful.  This  is  often  the  case,  because  people 
in  organizations  do  not  understand  what  effectiveness  is, 
nor  the  parameters  which  underlie  it. 

Once  people  in  an  organization  select  their  metrics, 
how  do  they  know  if  they  have  meaningful  measures?  That  is, 
how  do  they  know  that  what  they  are  measuring  will  help  them 
improve  quality?  People  desire  confidence  in  the  fact  that 
the  measures  being  used  mean  something  and  can  be  used  to 
control  the  processes  within  the  organizations  that  the 
metrics  measure.  Better  metrics  will  provide  managers  with 
better  information  to  make  better  decisions.  The  objective 
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of  this  thesis  is  to  provide  managers  with  a  tool  to 
evaluate  their  metrics,  identify  how  the  metrics  they 
possess  can  be  best  utilized,  and  identify  areas  where  they 
could  use  metrics  to  monitor  their  organization. 

Statement  of  the  Problem 

The  basic  statement  of  the  problem  is  that  people  have 
generated  many  measures  of  productivity,  but  there  seems  to 
be  a  low  level  of  confidence  in  the  purpose  or  use  of  the 
measurements  and  in  the  meaning  of  the  measures  to  the 
organization.  Certain  statistics  have  become  embedded  in 
management,  and  for  management  to  stray  from  using  these 
measures  would  not  be  "politically  acceptable"  (Stone, 

1995) .  In  general  the  measurements  that  are  being  used  may 
not  be  supported  by  or  built  on  substantiated  models  of 
organization  effectiveness.  They  may  not  be  part  of  a 
system  to  measure  the  organization;  they  may  simply  be 
measures  of  convenience  with  little  relationship  to 
effectiveness . 

Purpose 

The  purpose  of  this  thesis  is  to  develop  a  system  which 
can  be  used  to  place  metrics  of  an  organization  within  a 
substantiated  model  of  organizational  effectiveness.  The 
thesis  should  provide  organizations  a  way  to  evaluate 
metrics  against  a  standard  and  as  part  of  an  organization- 
specific  system. 
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Questions 

The  two  major  questions  this  thesis  seeks  to  answer 

are : 

1.  Is  there  an  adequate  model  available  on  which  metric 
evaluation  can  be  based? 

2 .  Can  an  instrument  be  developed  to  evaluate  metrics 
against  such  a  model? 

Assumptions 

The  first  assumption  of  this  thesis  is  that  aspects  of 
effectiveness  can  be  objectively  measured.  The  second 
assumption  is  that  each  organization's  effectiveness 
measurement  needs  are  unique  and,  therefore,  no  single  set 
of  measurements  will  be  appropriate  for  all  organizations. 
The  third  assumption  is  that  although  each  organization  is 
unique,  there  will  be  general  categories  into  which  metrics 
can  be  grouped.  The  fourth  assumption  is  that  an 
organization's  measurement  needs  can  be  identified.  The 
final  assumption  is  that  organizations  cannot  depend  on  a 
single  measurement  of  organization  effectiveness,  but 
rather,  must  depend  on  a  "constellation"  of  measurements  to 
judge  their  effectiveness. 

Definition  of  Terms 

Effectiveness  -  defined  as  the  degree  to  which  an 

organization  realizes  its  goals (Etzioni ,  1964:8). 
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Efficiency  -  Using  resources  to  their  maximum (Robbins , 
1990:77) . 

-  Accomplishment  of  or  the  ability  to  accomplish  a  job 
with  a  minimum  expenditure  of  time  and  effort (Stein, 
1980:421) . 

-  An  economic  index  of  the  ratio  of  measured  inputs  to 
measured  outputs (Cummings,  1983:198). 

-  Achievement  of  the  greatest  output  for  the  least 
input  (Mott,  1972:17). 

Measurement  -  The  act  or  process  of  quantitatively  comparing 
results  to  requirements  to  arrive  at  a  quantitative 
estimate  of  performance  (Holmes,  1994:108). 

Metrics  -  a  measurement,  taken  over  a  period  of  time,  that 
communicates  vital  information  about  a  process  or 
activity  (Holmes,  1994:108). 

-  meaningful  measures  that  present  data,  leading  to 
fact-based  decisions (Holmes,  1993:57). 

-  of,  relating  to  measurement (Stein,  1980:421). 

Organization  Effectiveness  -  the  degree  to  which  an 

organization  attains  its  short-term  and  long-term 
goals,  ends  and  means,  the  selection  of  which  reflects 
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strategic  constituencies  and  the  self-interest  of  the 
evaluator (Robbins,  1990:77). 

-  The  ability  of  an  organization  to  mobilize  its 
centers  of  power  for  action-production  and  adaptation 
(Mott,  1972:17). 

Quality  -  Consistently  meeting  or  exceeding  customer 
expectations  (Holmes.  1993:1). 

-  Fitness  for  use  (Devor,  1992)  . 

-  The  degree  to  which  a  product  satisfies  the  wants  of 
a  customer  (Evans,  1993)  . 

-  The  degree  to  which  a  product  conforms  to  design 
specification  and  engineering  requirements  (Evans, 

1993)  . 

Quality  Air  Force  (QAF)  -  The  Air  Force  approach  to  total 
quality  management :  a  leadership  commitment  and 
operating  style  that  inspires  trust,  teamwork  and 
continuous  improvement  everywhere  in  the  Air 
Force (Holmes .  1993:1). 

Total  Quality  -  A  strategically  integrated  system  for 
achieving  customer  satisfaction  that  involves  all 
managers  and  employees  and  uses  quantitative  methods  to 
continuously  improve  an  organization's  processes 
(Holmes,  1994:118). 
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Scope  and  Delimitation 

This  thesis  will  not  provide  the  one  best  measurement 
system  for  organizations  to  evaluate  all  of  their  metrics. 
The  thesis  will  develop  a  way  to  evaluate  metrics,  but  will 
not  validate  any  specific  measurements.  It  is  hoped  that 
the  thesis  will  provide  a  way  for  organizations  to  determine 
the  balance  of  their  measures,  but  it  is  not  going  to 
provide  the  one  best  set  of  measures  for  an  organization  to 
measure  its  effectiveness.  The  thesis  might  provide  a  way 
for  organizations  to  determine  the  purpose  of  a  metric,  but 
it  will  not  determine  the  purpose  of  any  particular  metrics. 

Summary 

This  chapter  established  the  focus  of  this  research 
effort.  It  outlines  the  general  background  of  the  questions 
and  outlines  the  questions  that  will  attempt  to  be  answered 
through  research.  Chapter  II  will  examine  current 
literature  to  assess  the  place  of  this  thesis  in  relation  to 
what  has  been  or  is  being  done  in  the  area  of  organizational 
effectiveness  research.  Chapter  III  will  outline  the 
methodology  that  will  be  implemented  to  answer  the  research 
questions.  Chapter  IV  will  analyze  the  results  of  the  data 
collected  through  the  methods  used.  Chapter  V  will  examine 
the  results  of  the  analysis  and  provide  recommendations  for 
future  research. 
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II.  Background 


Chapter  Overview 

Before  pursuing  the  research  objective  of  developing  an 
instrument  to  measure  metrics,  relevant  information 
concerning  organizational  effectiveness  and  some  of  the 
diverse  approaches  used  to  measure  it  will  be  examined.  The 
four  approaches  that  will  be  examined  are  the  goal 
attainment,  systems,  strategic  constituencies,  and  competing 
values  approaches .  Each  approach  will  be  examined  regarding 
its  definition,  assumptions,  and  problems.  Finally,  the 
four  approaches  will  be  compared  in  terms  of  value  to  the 
organization.  The  framework  for  this  chapter  is  similar  to 
one  used  by  Robbins  (1990)  wherein  he  analyzes  the 
assxamptions ,  problems,  and  value  to  managers  of  various 
organizational  effectiveness  models.  His  outline  and  style 
of  analysis  are  useful  in  evaluating  effectiveness  models 
and  have  been  accepted  as  a  model  for  this  chapter. 

Organizational  Effectiveness 

There  are  problems  inherent  in  defining  and  measuring 
organizational  effectiveness  (Robbins,  1990:48).  In  the 
past  researchers  have  found  it  difficult  to  determine  what 
organizational  effectiveness  means.  However,  most  agree 
that  organizational  effectiveness  is  central  to  organization 
theory.  In  fact,  it  is  difficult  to  conceive  of  a  theory  of 
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organizations  that  does  not  include  the  concept  of 
effectiveness  (Goodman  and  Pennings,  1977:2). 

Organizational  effectiveness  is  a  critical  concept  in 
organizational  theory  (Lawler  and  others,  1980:186). 

Effectiveness  is  important  to  organizations  because  it 
measures  its  capabilities,  its  ability  to  attain  its  goals, 
or  earning  potential.  Thus,  effectiveness  usually  relates 
directly  to  profitability  or  mission  effectiveness.  The 
three  most  important  criteria  for  determining  an 
organization's  effectiveness  are  quantity,  quality  and 
efficiency  (Mott,  1972:17).  Robbins  suggests  that  structure 
is  what  determines  an  organization's  effectiveness.  Thus, 
the  structure  of  the  organization's  people  and  jobs,  along 
with  their  roles  and  relationships,  are  important 
determinants  of  whether  an  organization  is  successful. 
Unfortunately,  there  is  not  a  universal  definition  of 
organizational  effectiveness.  One  reason  there  are  no  best 
criteria  for  effectiveness  is  because  there  is  no  best 
constituency  to  define  effectiveness.  (Cameron  and  Whetten, 
1983:270).  For  example,  effectiveness  to  stock  holders  is 
likely  to  be  somewhat  different  than  effectiveness  to 
management.  No  agreed  upon  decision  rule  is  available  to 
identify  one  constituency's  criteria  as  being  more  important 
than  another  constituency's  criteria  because  it  partly 
depends  on  the  other  decision  guides  and  partly  on  personal 
biases  of  the  evaluator.  Nevertheless,  organizations  rarely 
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satisfy  all  their  constituencies,  and  what  appears  to  be 
high  effectiveness  from  one  point  of  view  may  be  interpreted 
as  being  mediocre  or  low  effectiveness  from  another  point  of 
view  (Cameron  and  Whetten,  1983:270). 

The  early  approaches  defined  effectiveness  as  the 
degree  to  which  an  organization  realized  its  goals  (Etzioni, 
1964:8).  Hidden  in  this  definition,  however,  were  many 
ambiguities  that  severely  curtailed  both  research  on  the 
subject  and  practicing  managers'  ability  to  grasp  and  use 
the  concept  (Robbins,  1990:49).  One  condition  most 
researchers  agree  is  necessary  for  an  organization's  success 
is  survival  (Kimberly,  1979:438).  Of  course,  this  assumes 
that  we  can  identify  the  death  of  an  organization.  In 
reality  most  organizations  merge,  sell  off  parts  or 
liquidate,  but  no  actual  "death"  is  identifiable. 
Additionally  if  an  organization  is  its  people,  then  the 
members  of  an  organization  rarely  die  when  the  organization 
fails  to  exist.  For  some  organizations  death  practically 
never  occurs,  while  in  some  cases,  the  organization  commits 
suicide  to  escape  eventual  death  in  the  most  effective 
manner  (Pfeffer,  1978) .  Most  important  of  all,  we  must 
recognize  that  even  a  commonly  agreed  upon  term  like 
survival,  collapses  under  close  analysis. 

The  belief  that  organizational  effectiveness  defies 
definition  has  been  widely  accepted  (Robbins,  1990:51). 
However,  this  belief  is  mainly  due  to  the  previous  research 
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done  on  the  subject  highlighting  the  differences  or 
weaknesses  among  organizational  effectiveness  models. 
Previous  writing  has  largely  aimed  at  replacing  older 
perspectives  on  effectiveness  rather  than  adding  to  them 
(Cameron  and  Whetten,  1983:261).  The  more  recent  studies 
are  starting  to  examine  the  similarities  or  commonalities  of 
organizational  effectiveness  criteria  (Cameron,  1986:539). 

It  is  widely  agreed  that  organizational  effectiveness 
requires  multiple  criteria,  that  different  organizational 
functions  have  to  be  evaluated  using  different 
characteristics,  and  organizational  effectiveness  must 
consider  both  means  (process)  and  ends  (outcomes)  (Robbins, 
1990:51).  We  still  have  to  wait  for  some  gifted  theorist 
who  can  develop  a  paradigm  that  can  unify  the  present 
accomplishments  and  provide  theoretical  guidance  for  those 
who  wish  to  tackle  this  construct  (Pennings,  1978:539). 

Organizational  assessment  models  are  basically  theories 
of  organizational  functioning  (Lawler  and  others,  1980:121). 
As  theories  they  display  certain  characteristics  (Kerlinger, 
1964) .  Organization  theories  typically  contain  a  particular 
construct  called  effectiveness.  This  provides  a  means  for 
evaluating  the  performances  of  an  organizational  system,  not 
just  describing  it  (Lawler  and  others,  1980:121). 

Using  a  model  for  assessing  an  organizations' 
effectiveness  effort  can  provide  many  benefits. 
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At  the  most  simplistic  level,  using  a  model  facilitates 
communication  among  the  evaluators,  who  will  be  able  to 
approach  their  task  with  a  common  set  of  terms  and 
frame  of  reference  and  avoid  confusing,  nonproductive, 
and  personalized  abstractions  of  the  domain  to  be 
covered.  Indeed,  if  the  model  can  be  represented 
graphically,  it  becomes  a  "map"  of  that  domain  to  which 
everyone  can  refer  and  return  when  necessary.  Further, 
the  model  makes  clear  the  factors  and  relationships 
that  are  of  interest  and,  by  implication,  those  that 
are  not.  (Lawler  and  others,  1980:133) 

A  model  is  perhaps  most  beneficial  in  that  it  defines  what 

should  be  assessed.  One  way  to  understand  the  problem  of 

specifying  the  domain  of  organizational  effectiveness  is  to 

briefly  examine  the  indicators  used  in  organizational 

effectiveness  studies.  Campbell  provides  an  outline  for 

understanding  these  indicators  (Table  2-1) . 


Table  1.  Top  30  Organizational  Effectiveness  Indicators 


Overall  effectiveness 

Motivation 

Managerial  task  skills 

Productivity 

Morale 

Information  management 

Efficiency 

Control 

Readiness 

Profit 

Conflict-cohesion 

Utilization  of  the 
environment 

Quality 

Flexibility- adaptation 

Evaluation  by  external 
entities 

Accidents 

Planning  and  goal 
setting 

Stability 

Growth 

Goal  consensus 

Value  of  human 

resources 

Absenteeism 

Internalization  of 
goals 

Participation  and 
shared  influence 

Turnover 

Role  and  norm 
congruence 

Training  and 
development  emphasis 

Job  satisfaction 

Managerial 
interpersonal  skills 

Achievement  emphasis 

(Campbell  and  others,  1974) 


What  can  be  learned  from  this  list?  First  although  the  list 
is  long,  it  is  not  exhaustive.  We  can  think  of  other 
criteria  which  might  be  used  to  measure  organizational 
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effectiveness.  For  example,  stock  prices  or  company- 
publications.  Second,  not  all  of  the  indicators  are  likely 
to  be  relevant  for  all  types  of  organizations.  For  example, 
service  versus  product  oriented  firms  and  small  versus  large 
firms  would  probably  have  different  indicator  lists. 
Additionally,  some  of  the  indicators  may  be  determinants, 
that  is,  causes  of  other  indicators  (Lawler  and  others, 
1980:194).  The  remainder  of  this  chapter  attempts  to 
outline  some  of  the  diverse  approaches  to  studying 
organizational  effectiveness  that  utilize  indicators  to 
evaluate  organizational  effectiveness. 

The  Goal  Attainment  Approach 

An  organization  is,  by  definition,  created  deliberately 

to  achieve  one  or  more  specified  goals  (Perrow,  1961:854). 

Thus,  the  most  widely  used  method  to  measure  organization 

effectiveness  is  the  goal  attainment  approach.  An  example 

of  a  common  goal  attainment  approach  is  called  "Management 

by  Objective"  (MBO)  and  is  defined  as: 

A  management  process  whereby  the  supervisor  and  the 
subordinate,  operating  under  a  clear  definition  of  the 
common  goals  and  priorities  of  the  organization 
established  by  top  management,  jointly  identify  areas 
of  responsibility  in  terms  of  results  expected  and  use 
these  measures  as  guides  for  operating  the  unit  and 
assessing  the  contributions  of  each  of  its  members. 
(Odiorne,  1965:55-56) 

The  goal  attainment  approach  focuses  on  the  ends  rather 
than  the  means  or  process.  It  is  profit  maximization, 
growth,  increased  productivity,  the  bottom  line,  winning  the 
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game  or  war,  reduction  of  disease  or  restoring  a  patient's 

health  that  counts.  The  commonality  is  that  the  end  for 

which  the  organization  was  created  is  the  primary 

consideration.  In  these  terms,  the  organization  that 

attains  some  predetermined  objective,  purpose,  mission  or 

goal  (usually  an  outcome  or  end)  is  operating  effectively. 

The  implication  is  that  organizations  are  goal-seeking 

entities  which  can  most  rationally  be  evaluated  by  measuring 

their  degrees  of  goal  attainment  (Strasser,  1978:6). 

Assumptions .  The  underlying  assumption  is  that 

organizations  are  goal-seeking  entities.  Thus,  goal 

accomplishment  is  the  measurement  of  choice  for  measuring 

organizational  effectiveness.  However,  the  use  of  goals 

requires  other  assumptions  to  be  present  if  goal 

accomplishment  is  to  be  a  useful  measurement. 

First,  an  organization  needs  to  be  able  to  define 
ultimate  goals.  Second,  these  goals  need  to  be  defined 
so  that  all  involved  can  understand  them.  Third,  the 
number  of  goals  needs  to  be  few  enough  to  be 
manageable.  Fourth,  a  majority  of  the  organizational 
leaders  need  to  have  a  general  consensus  on  the  goals . 
Finally,  progress  toward  achieving  these  goals  needs  to 
be  measurable.  (Robbins,  1990:54) 

Problems .  The  goal  attainment  approach  has  several 
problems  that  would  deter  a  manager  from  using  it  as  the 
only  approach.  Many  of  the  problems  relate  directly  to  the 
previously  mentioned  assumptions. 

When  the  goal  attainment  approach  is  put  into  use,  one 
must  ask,  "Whose  goals?  Management's?"  If  so,  what  is  the 


14 


criteria  for  being  included  as  a  goal  designed  for  the 
organization?  Also,  often  members  of  an  organization  who 
have  the  most  influence  on  the  organization's  goals  are  not 
part  of  management.  Etzioni,  for  example,  argues  that  the 
goal  model  evaluator  often  "projects"  his  own  value  system 
into  the  goal  identification  process  (1960:112).  The  result 
is  an  evaluation  of  the  evaluator's  goals  and  not  the 
organization's. 

Katz  and  Kahn  apply  the  "Whose  goal  critique"  in  two 
different  ways.  On  one  level  an  invalid  set  of 
organizational  goals  might  be  identified  by  the  evaluator, 
because  his  source  of  information  in  the  organization  "may 
idealize,  rationalize,  distort,  omit  or  even  conceal  some 
essential  aspects  of  the  functioning  of  the  organization" . 

On  a  second  level  they  suggest  that  the  goal  model  confuses 
"purposes  or  goals  of  organizations  with  the  purposes  or 
goals  of  individual  members"  (Katz  and  Kahn,  1966:15) .  One 
implication  of  this  is  that  the  evaluator  may  focus 
attention  on  a  non-representative  set  of  goals,  that  is  a 
set  of  goals  of  only  the  top  management,  and  not  that  of  the 
organizational  members.  In  the  first  case  the  outcome  can 
be  an  evaluation  of  false  goals;  in  the  second,  a  very 
limited  scope  evaluation  may  result. 

Official  goals  of  an  organization  are  often  not  the 
organizations  actual  goals  (Warriner,  1965:140).  Official 
goals  of  an  organization  tend  to  be  influenced  by  what  is 


15 


socially  acceptable  to  society,  or  stakeholders.  Often, 
some  of  the  goals  stated,  such  as  to  hire  minorities  or 
protect  the  environment,  do  not  tell  us  what  the 
organization  is  actually  trying  to  accomplish  or  they 
exclude  measures  of  organization  behavior  and  processes. 
Thus,  when  evaluating  an  organization's  goals,  one  should 
evaluate  both  what  the  organization  says  are  their  goals  and 
what  the  members  of  the  organization  are  actually  doing 
(Robbins,  1990:55).  Similarly,  an  organization's  short-term 
goals  are  often  different  from  its  long-term  goals.  The 
problem  that  arises  from  this,  is  the  dilemma  an 
organization  has  in  choosing  which  goals  to  implement, 
short-term  or  long-term. 

It  may  just  be  that  for  many  organizations,  goals  do  not 

direct  behavior  (Robbins,  1990:56). 

The  common  assertion  that  goal  consensus  must  occur 
prior  to  action  obscures  the  fact  that  consensus  is 
impossible  unless  there  is  something  tangible  around 
which  it  can  occur.  And  this  something  tangible  may 
well  turn  out  to  be  actions  already  completed.  (Weick, 
1969:8) 

In  summary,  the  goals  an  organization  presents  depend  on 
whom  they  present  the  goals  to.  Corporations  issue  one  set 
of  goals  to  stockholders,  another  to  customers,  a  third  set 
to  employees,  a  fourth  to  the  public,  and  still  a  fifth  set 
for  management  itself.  Formal  statements  of  goals  should  be 
treated  as  fiction  produced  by  an  organization  to  account 
for,  explain,  or  rationalize  its  existence  to  particular 
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audiences  rather  than  as  valid  and  reliable  indications  of 


purpose  (Warriner,  1965:143). 

Value  to  Managers.  These  problems,  while  certainly- 
required  for  consideration  of  instituting  the  goal 
attainment  approach,  they  are  not  intended  to  condemn  the 
usefulness  of  the  approach.  Goal  models  are  more  objective 
than  some  of  the  other  models.  Additionally,  their  outputs 
are  easy  to  interpret.  Organizations  exist  to  achieve 
goals--the  problems  lie  in  their  identification  and 
measurement . 

The  validity  of  those  goals  identified  can  probably  be 
increased  significantly  by  (1)  ensuring  that  input  is 
received  from  all  these  having  a  major  influence  on 
formulating  the  official  goals,  even  if  they  are  not 
part  of  senior  management;  (2)  including  actual  goals 
obtained  by  observing  the  behavior  of  organization 
members'  (3)  recognizing  that  organizations  pursue  both 
short-  and  long-term  goals;  (4)  insisting  on  tangible, 
verifiable  goals  rather  than  relying  on  vague 
statements  that  merely  mirror  societal  expectations; 
and  (5)  viewing  goals  as  dynamic  entities  that  change 
over  time  rather  than  as  rigid  or  fixed  statements  of 
purpose.  (Robbins,  1990:57) 

If  managers  are  willing  to  tackle  the  inherent  problems  in 
the  goal  attainment  approach,  they  can  obtain  valuable 
information  in  measuring  an  organizations'  effectiveness. 
However,  there  is  more  to  organizational  effectiveness  than 
establishing  goals  and  measuring  specific  ends. 


The  Systems  Approach 

One  method  of  looking  at  organizations  is  as  a  system, 
with  inputs,  processes  to  transform  the  inputs,  and  outputs. 
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A  cooperative  system  is  a  complex  of  physical, 
biological,  personal,  and  social  components  which  are 
in  a  specific  systematic  relationship  by  reason  of  the 
cooperation  of  two  or  more  persons  for  at  least  one 
definite  end.  Such  a  system  is  evidently  a  subordinate 
unit  of  larger  systems  from  one  point  of  view;  and 
itself  embraces  subsidiary  systems-physical , 
biological,  etc. -from  another  point  of  view.  One  of 
the  systems  comprised  within  a  cooperative  system,  the 
one  which  is  implicit  in  the  phrase  "cooperative  of  two 
or  more  persons,"  is  called  an  organization.  (Barnard, 
1938:65) 

A  system,  by  its  very  nature,  is  made  up  of  interdependent 
elements.  As  such,  actions  that  affect  one  element  must 
affect  others  also.  Cleland  states  that  actions  of  one 
element  cause  reactions  on  the  part  of  others  (1972) .  End 
goals,  outputs,  are  not  ignored,  but  are  part  of  the  system 
as  a  whole.  Thus,  the  approach  focuses  more  on  the  means  to 
achieving  goals,  than  the  goals  themselves.  The  recognition 
of  such  interactions  and  interdependencies  both  within  and 
without  the  organization  is  the  essence  of  the  systems 
viewpoint  (Cleland,  1972:77). 

Assumptions .  The  first  assumption  is  that  organizations 
are  made  of  entities,  working  together  to  accomplish  a  goal. 
These  entities  are  related,  that  is  if  any  one  of  the 
entity's  performance  is  poor,  it  will  likely  have  a  negative 
effect  on  the  organization  as  a  whole.  The  second 
assumption  is  that  organizations  can  be  measured  by 
effectiveness  objectively.  The  third  assumption  is  that 
organizations  must  maintain  a  working  relationship  with 
external  entities  to  survive.  That  is,  if  they  do  not 
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maintain  a  working  relationship  with  external  entities,  then 
the  organization's  operational  effectiveness  will  be 
affected.  The  fourth  assumption  is  that  an  organization 
cannot  survive  without  steady  replenishment  of  resources, 
including  the  replacement  of  personnel,  adapting  to  the 
environment,  and  replacement  of  subpar  outputs  of  the 
organization.  Failure  to  replenish  resources  will  result  in 
the  organizations'  decline  and,  possibly,  death  (Robbins, 
1990:59) . 

Problems .  Critics  consider  the  systems  approach 
abstract  and  not  very  practical  (Cleland,  1972) .  Talking 
about  inputs,  transformations,  and  outputs  isn't  how 
everyday  managers  discuss  problems,  make  decisions,  and  face 
reality  (Ivancevich  and  others,  1994:57).  Two  of  the  more 
prominent  problems  of  the  systems  approach  are  measuring  and 
the  point  of  whether  the  means  really  matter.  It  is  argued 
that  the  most  important  measure  of  an  organization  is 
survival,  so  does  it  really  matter  how  they  survive.  The 
goal  is  to  survive,  not  to  look  good  while  dying.  Measuring 
specific  end  goals  may  be  easy  compared  with  trying  to 
measure  process  variables  such  as  "flexibility  of  response 
to  environmental  change"  or  "clarity  of  internal 
communications"  (Robbins,  1990:61).  Developing  measures 
that  are  valid  and  reliable  to  measure  processes  as  these 
may  be  impossible.  One  example  is  transportation  planning, 
where  success  has  been  achieved  by  saving  time  for 
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passengers.  The  question  is,  What  is  time  worth?  In  fact, 
more  complex  considerations  arise,  are  savings  of  one  hour 
for  a  single  individual  equal  to  sixty-one  minute  time 
savings  for  sixty  people?  The  problem  with  measuring  means 
is  that  the  concern  is  with  how  organizational  effectiveness 
is  achieved,  rather  than  whether  or  not  an  organization  is 
effective.  The  focus  is  still  on  achieving  a  single  goal  of 
accomplishing  certain  means.  This  is  sub  optimization 
because  it  only  focuses  on  part  of  the  organization. 

Value  to  Managers .  The  existence  of  clearly  defined 
objectives  for  an  organization  has  value.  First,  people 
know  what  they  should  be  trying  to  do,  and  it  is  thereby 
possible  for  all  to  work  toward  a  common  goal.  Second,  clear 
objectives  serve  as  a  basis  for  performance  evaluation  and 
control  (Cleland,  1972:234).  Managers  who  use  this  approach 
tend  to  optimize  the  health  of  the  organization  for  long 
term  survival  as  opposed  to  looking  good  today.  Managers 
tend  to  be  more  aware  of  how  one  element  of  the  organization 
relates  to  another.  For  instance,  if  management  overlooks 
raw  materials  and  there  is  a  shortage,  then  the  organization 
is  less  likely  to  achieve  its  end  goals.  Another  plus  for 
the  systems  approach  is  its  applicability  where  end  goals 
either  are  very  vague  or  defy  measurement  (Robbins, 

1990:62) .  Managers  often  use  budget  increases  as  a 
measurement  of  effectiveness,  thus  substituting  a  raw 
material  measurement  for  an  output  measurement. 
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The  Strategic  Constituencies  Approach 

The  strategic  constituencies  approach  proposes  that  an 
effective  organization  is  one  that  satisfies  the  demands  of 
those  constituencies  in  its  environment  from  whom  it 
requires  support  for  its  continued  existence  (Pfeffer  and 
Salancik,  1978) . 

The  contingency  view  of  organizations  and  their 
management  suggests  that  the  organization  is  a  system 
composed  of  subsystems  and  delineated  by  identifiable 
boundaries  from  its  environmental  suprasystem.  The 
contingency  view  seeks  to  understand  the 
interrelationships  within  and  among  subsystems,  as  well 
as  between  the  organizations  of  variables.  It 
emphasizes  the  multivariate  nature  of  organizations  and 
attempts  to  understand  how  organizations  operate  under 
varying  conditions  and  in  specific  circumstances. 
Contingency  views  are  ultimately  directed  toward 
suggesting  organizational  designs  and  managerial 
systems  most  appropriate  for  specific  situations. 
(Ivancevich  and  others,  1994:58) 

A  constituent  or  stakeholder  is  defined  as  any  group  or 

individual  who  can  affect  or  is  affected  by  the  achievement 

of  the  organizations  objectives  (Thompson,  1967) .  The 

underlying  approach  is  similar  to  that  of  the  systems 

approach,  with  one  exception.  The  strategic  constituencies 

approach  is  not  concerned  with  all  of  the  organizations’ 

environment,  but  rather  only  those  constituents  that  can 

directly  effect  the  survival  of  the  organization.  The  issue 

is  pure  and  simple:  survival  (Freeman,  1984:33).  That  is 

the  constituents  that  need  to  be  kept  happy  or  satisfied  in 

order  for  the  organization  to  continue  to  thrive.  Some 
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constituents  or  stakeholders  must  be  satisfied  in  order  to 


keep  them  from  disrupting  the  smooth  operations  of  an 
organization . 

For  instance,  some  corporations  must  count  "terrorist 
groups"  as  stakeholders.  As  unsavory  as  it  is  to  admit 
that  such  "illegitimate"  groups  have  a  stake  in  our 
business,  from  the  standpoint  of  strategic  management, 
it  must  be  done.  Strategies  must  be  put  in  place  to 
deal  with  terrorist  if  they  can  substantially  effect 
the  operations  of  the  business.  (Freeman,  1984:53) 

Additionally,  the  military  serves  to  protect  the  public,  but 

the  constituent  that  controls  its  existence  is  Congress, 

thus  the  military  attempts  to  satisfy  Congress'  desires 

instead  of  the  general  public's. 

Assumptions .  The  strategic  constituencies  approach 

views  organizations  as  political  arenas  where  different 

stakeholders  compete  for  control  over  various  resources 

(Robbins,  1990:63).  Thus,  the  measurement  of  effectiveness 

is  the  level  to  which  an  organization  satisfies  its  critical 

constituents,  at  least  those  from  which  the  organization's 

survival  depends  upon.  It  also  assumes  that  the  stakeholder 

environment  is  static  and  that  there  will  be  no  radical 

shifts  in  the  stakeholders  actions  (Freeman,  1984:35). 

Because  the  stakeholder  environment  is  static,  adversarial 

groups  are  not  considered  as  stakeholders  (Freeman, 

1984:35).  Additionally  the  approach  assumes  that  there  are 

different  stakeholders,  with  various  levels  of  influence, 

each  trying  to  satisfy  its  own  demands.  However,  each 

stakeholder  has  its  own  demands,  usually  different  from  the 
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organization's.  Finally,  the  strategic  constituencies 
approach  assumes  that  the  goals  pursued  by  the 
organization's  managers  represent  the  interest  of  the 
stakeholders  that  control  the  organizations  survival. 
Regardless  of  which  goals  are  pursued,  some  of  the 
constituents  will  be  more  satisfied  than  others. 

Problems .  As  with  other  approaches,  the  strategic 
constituencies  approach  has  its  problems.  The  task  of 
separating  the  strategic  constituencies  from  the  larger 
environment  is  easy  to  say  but  difficult  to  do  in  practice 
(Robbins,  1990:67).  Due  to  a  rapid  changing  environment, 
what  defines  who  the  strategic  stakeholders  are  today,  as 
opposed  to  yesterday?  More  importantly,  the  managers  who  are 
defining  the  critical  stakeholders  may  have  different 
opinions  or  interests,  as  to  who  the  critical  stakeholders 
are,  among  themselves.  If  the  organization  strictly  focuses 
its  priorities  toward  the  constituents  identified  today, 
then  it  may  not  identify  new  critical  constituents  until 
after  they  have  already  impacted  the  organization.  The 
organization  may  not  take  into  account  the  impact  some  of 
the  less  critical  constituents  may  have  in  the  future. 
Additionally,  the  concerns  or  goals  of  the  organization  may 
not  be  in  line  with  its  most  critical  stakeholder,  causing 
the  organization  to  risk  its  demise.  The  organization  might 
pursue  a  direction  for  the  organization  that  is  not  in  its 
best  interest.  The  smaller,  less  critical  stakeholders  are 
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often  overlooked.  However,  they  may,  all  of  a  sudden, 
become  important  due  to  a  change  in  the  environment . 

Finally  determining  what ' the  stakeholders  actually  expect 
the  organization  to  provide  them  can  be  difficult  at  best. 

Value  to  Managers,  if  the  sole  interest  of  management  is 
survival,  then  it  is  inherent  that  management  knows  who  its 
stakeholders  are.  If  an  organization  implements  the 
strategic  constituencies  approach,  the  possibility  of  it 
overlooking  a  stakeholder  on  which  its  survival  depends  is 
less  likely.  While  at  the  same  time,  the  organization  can 
build  a  knowledge  base  on  particular  stakeholders  for  use  in 
future  management  decisions.  As  a  result,  the  approach 
provides  the  organization  with  concrete  analytical  tools  and 
managerial  processes.  Additionally,  by  regularly 
identifying  the  stakeholders  required  for  the  organization's 
survival,  the  organization  can  manipulate  its  goals  and 
priorities  to  meet  the  demands  of  the  constituent  that  has 
the  most  influence  or  power  over  the  organization. 

The  Competing  Values  Approach 

Several  studies  that  address  the  characteristics  of 
effective  organizations  have  been  performed  (Quinn, 

1988:47)  .  The  results  vary  depending  on  which  study  is 
examined.  Thus,  Quinn  and  Rohrbaugh  developed  the  competing 
values  model  to  analyze  what  experts  think  about  effective 
organizations  (1981:122). 
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Instead  of  asking  what  effective  organizations  looked 
like,  we  decided  to  ask  how  experts  think  about 
effective  organizations.  This  would  allow  us  to  get  to 
the  assumption  behind  the  studies  and  perhaps  make 
sense  of  what  was  causing  the  confusion.  The  data  from 
the  studies  were  analyzed  using  a  technique  called 
multidimensional  scaling.  Results  of  the  analyses 
suggested  that  organizational  theorists  and  researchers 
share  an  implicit  theoretical  framework,  or  cognitive 
map.  (Figure  1)  (Quinn,  1988:47) 


Competing  Values 


Autonomy 

Decentralization 


People 


Organization 


Figure  1.  Competing  Values  Framework. 


Figure  1  shows  the  three  "multidimensional  scales" 
Quinn  and  Rohrbaugh  identified  as  being  relevant  to 
organizational  effectiveness.  The  vertical  axis  represents 
decentralization  on  one  end  and  centralization  at  the  other, 
while  the  horizontal  axis  represents  people  as  the  basic 
subelement  of  the  organization  on  one  end  and  the 
organization  as  a  whole  on  the  other.  The  axis  between  the 
horizontal  and  vertical  axis  represents  means  or  process  on 
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one  end  and  ends  or  goals  on  the  other.  This  framework  has 
been  expanded  (Figure  2)  to  form  a  more  constructive 
framework  for  measuring  organizational  effectiveness. 
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Figure  2.  Competing  Values  Framework:  Effectiveness 


The  vertical  axis  represents  flexibility  and  autonomy 
at  one  end  and  stability  and  control  at  the  other.  The 
horizontal  axis  represents  internal  focus  on  individuals  on 
one  end  and  a  more  external  focus  on  the  whole  organization 
on  the  other.  The  intersection  of  these  two  axis  forms  four 
quadrants,  with  each  quadrant  representing  one  of  the  four 
major  models  in  organization  theory.  The  upper  left 
quadrant  identifies  with  the  humans  relations  model  and 
emphasizes  morale  and  human  resource  development.  The  upper 
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right  quadrant  identifies  with  the  open  systems  model  and 
emphasizes  flexibility,  readiness,  growth  and  external 
support.  The  lower  left  quadrant  identifies  with  the 
internal  process  model  and  emphasizes  information 
management,  communication,  and  bureaucratization.  The  lower 
right  quadrant  identifies  with  the  rational  goal  model  and 
emphasizes  planning,  goal  setting  and  productivity. 

Each  model  has  a  polar  opposite  (Quinn,  1988:47).  For 
instance,  the  open  systems  model,  which  emphasizes 
flexibility  and  external  support,  stands  opposite  the 
internal  process  model,  which  emphasizes  control  and 
internal  focus.  Additionally,  there  are  parallels  among  the 
models .  Human  relations  and  open  systems  models  both  place 
an  emphasis  on  flexibility.  There  are  other  similarities 
among  the  other  models .  Each  of  the  models  has  a  means  to 
achieve  an  end.  For  instance,  "in  the  internal  process 
model  is  the  hierarchy,  where  information  management  and 
communication  are  viewed  as  a  means  of  arriving  at  stability 
and  control"  (Quinn,  1988:47) . 

The  scheme  is  called  the  competing  values 
framework  because  the  criteria  seem  to  initially  carry 
a  conflictual  message.  We  want  our  organizations  to  be 
adaptable  and  flexible,  but  we  also  want  them  to  be 
stable  and  controlled.  We  want  growth,  resource 
acquisition,  and  external  support,  but  we  also  want 
tight  information  management  and  formal  communication. 
We  want  an  emphasis  on  value  of  human  resources,  but  we 
also  want  an  emphasis  on  planning  and  goal  setting. 

The  model  does  not  suggest  that  these  oppositions 
cannot  mutually  exist  in  a  real  system.  It  suggests, 
rather,  that  these  criteria,  values,  and  assumptions 
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are  oppositions  in  our  minds.  We  tend  to  think  that 
they  are  very  different  from  one  another,  and  we 
sometimes  assume  them  to  be  mutually  exclusive.  (Quinn, 
1988:47-49) 

Quinn  has  applied  the  competing  values  model  to  many 
different  problems  in  the  past  (Quinn,  1988:49).  This  model 
provides  us  with  a  comprehensive  understanding  and  an 
integrated  framework  for  examining  organizational 
effectiveness . 

The  main  theme  underlying  the  competing  values  approach 
is  the  criteria  you  value  and  use  in  assessing  an 
organization's  effectiveness-  return  on  investment,  market 
share,  new  product  innovation  -  depend  on  who  you  are  and 
the  interests  you  represent  (Robbins,  1990:68).  It  is  not 
surprising  that  different  groups  of  people,  within  or 
outside  the  organization,  have  different  expectations  of  how 
an  organization  will  be  effective  in  meeting  their  needs. 
This  means  the  perspective  of  the  evaluator  can  determine 
what  the  values  are  most  important  to  be  effective.  The 
rating,  therefore,  probably  tells  us  more  about  the  values 
of  the  evaluator  (what  he  or  she  prefers  in  terms  of  an 
organization)  than  it  tells  us  about  the  organization's 
effectiveness  (Robbins,  1990:68). 

Assumptions .  it  begins  with  the  assumption  that  there 
is  no  "best"  criterion  for  evaluating  an  organization's 
effectiveness  (Robbins,  1990:68).  The  organization  cannot 
decide  on  a  single  goal  or  come  to  an  agreement  on  the 
priority  of  the  goals.  That  is,  because  the  managers  each 
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have  their  own  interests  and  values,  their  beliefs  in  what 
makes  the  organization  effective  differ. 

The  competing  values  model  recognizes  these  differences 
and  attempts  to  consolidate  and  organize  them.  The 
competing  values  approach  argues  that  there  are  common 
elements  underlying  any  comprehensive  list  of  organizational 
effectiveness  criteria  and  that  these  elements  can  be 
combined  in  such  a  way  as  to  create  basic  sets  of  competing 
values  (Robbins,  1990:69).  The  collection  of  these  elements 
then  forms  a  unique  effectiveness  model  for  the 
organization. 

Problems .  since  the  competing  values  model  integrates 
all  or  parts  of  the  other  models,  it  should  be  more 
effective  than  using  one  of  the  others  alone.  However, 
although  it  includes  the  constituencies  approach  it  does 
little  to  alleviate  the  problems  addressed  with  that 
approach  (Robbins,  1990:76).  Additionally,  the  approach 
does  more  for  identifying  management's  perception  on  how 
well  the  organization  is  doing  on  the  eight  different 
criteria  than  which  criteria  the  organization  is  emphasizing 
in  its  management  practices . 

Value  to  Managers .  Competing  values  recognizes  that 
managers  have  different  criteria  and  interest  when  they 
define  or  assess  their  organization's  effectiveness. 
Additionally,  by  reducing  a  large  number  of  effectiveness 
criteria  into  a  conceptually  clear  organizational  model,  the 
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competing  values  approach  can  guide  the  manager  in 
identifying  the  appropriateness  of  different  criteria  to 
different  constituencies  (Robbins,  1990:76). 

Comparing  the  Four  Approaches 

Four  different  approaches  for  assessing  organizational 
effectiveness  has  been  presented.  Each  of  the  models  is 
useful  in  its  own  way.  Table  2  illustrates  what  each 
approach  uses  to  define  effectiveness  and  suggests  when  it 
might  be  most  effective  at  measuring  organizational 
effectiveness . 

It  is  important  to  note  that  an  organization  is  not 
"located"  in  any  specific  quadrant  of  the  model.  Rather,  an 
organization  is  "mapped"  onto  the  Framework  to  show  the 
organizations  relative  emphasis  on  each  of  the  four 
effectiveness  quadrants. 
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Table  2.  Comparing  the  Four  Organizational  Effectiveness 

Approaches 


APPROACHES 

DEFINITION 

WHEN  USEFUL 

An  organization  is  effective 
to  the  extent  that  •  .  • 

The  approach  is  preferred  when 

Goal  Attainment 

it  accomplishes  its  stated 
goals . 

goals  are  clear,  time  bound, 
and  measurable . 

Systems 

it  acquires  needed 
resources . 

a  clear  connection  exists 
between  inputs  and  outputs. 

Strategic 

Constituencies 

all  strategic  constituencies 
are  at  least  minimally 
satisfied. 

constituencies  have  powerful 
influence  on  the  organization, 
and  the  organization  must 
respond  to  demands . 

Competing  Values 

the  emphasis  of  the 
organization  in  the  four 
major  areas  matches 
constituent  preferences. 

the  organization  is  unclear 
about  its  own  emphasis,  or 
changes  in  criteria  over  time 
are  of  interest. 

(Cameron,  1984:276) 


Summary 

Organizational  effectiveness  is  difficult,  some  experts 
say  impossible,  to  define.  Yet,  it  is  the  core  focus  of 
most  organizations'  management  in  understanding  their 
organization.  Four  approaches  have  been  examined  from  the 
organizational  effectiveness  model  "swamp." 

While  the  goal  attainment  approach  defines 
organizational  effectiveness  as  the  attainment  of  ends,  the 
systems  approach  focuses  on  means .  The  strategic 
constituencies  approach  provides  an  additional  perspective 
by  defining  organizational  effectiveness  as  satisfying  the 
demands  of  the  constituents  that  the  organization  must 
satisfy  to  remain  in  existence.  The  final  approach  was 
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competing  values,  which  focuses  on  both  means  and  ends, 
while  recognizing  its  constituents. 

Although  I  was  not  able  to  define  organizational 
effectiveness  more  precisely  than  has  been  done  in  the  past, 
I  was  able  to  find  a  model  of  organizational  effectiveness 
that  covers  a  wide  range  of  effectiveness  concepts  and 
parameters.  I  have  chosen  the  Competing  Values  Model  as  a 
basis  for  my  instriiment.  The  Competing  Values  Approach 
seems  to  encompass  more  of  the  indicators  of  effectiveness 
and  the  dimensions  that  are  required  to  truly  measure 
effectiveness . 
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III.  Methodology 


Chapter  Overview 

The  goal  of  this  research  was  to  develop  an  instrument 
capable  of  evaluating  an  organization's  metrics.  This 
chapter  describes  and  explains  the  methodology  used  to 
construct,  test,  and  interpret  the  results  of  the 
questionnaire. 

This  Chapter  begins  by  describing  the  rationale  for 
each  of  the  author's  objectives: 

1.  Is  there  an  adequate  model  available  on  which 
metric  evaluation  can  be  based? 

2 .  Can  an  instrument  be  developed  to  evaluate  metrics 
against  such  a  model? 

Restatement  of  Research  Objectives 

The  first  objective  was  to  select  an  adequate  model  or 
models  on  which  metric  evaluation  could  be  based.  Chapter 
II  accomplished  this  by  outlining  some  of  the  more  prominent 
models  available.  The  chapter  then  analyzed  each  model  for 
its  value  at  evaluating  an  organization's  effectiveness. 

The  model  that  was  selected  from  the  four  models  that  were 
examined  was  the  Competing  Values  Model.  A  complete 
explanation  of  that  model  is  included  in  Chapter  II. 

The  second  objective  was  to  create  an  instrument  to 
evaluate  metrics  that  measure  an  organization's 
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effectiveness  using  the  Competing  Values  Model.  The  reader 
will  find  a  discussion  of  the  methods  used  and  conclusions 
of  this  portion  of  the  research  in  this  chapter  and  Chapter 
IV,  Data  Analysis. 

Research  Approach 

The  most  appropriate  method  for  conducting  this  study 
was  determined  to  be  an  instrument  validation  methodology 
using  a  survey.  As  reflected  by  the  research  objectives  and 
the  literature  review,  the  evaluation  of  metrics  within  an 
organization  for  their  effectiveness  in  measuring 
organizational  effectiveness  is  in  its  infancy.  Thus,  the 
instrument  was  based  on  an  existing  model,  using  questions 
based  on  Competing  Values  Model  parameters  to  validate  the 
ability  to  differentiate  among  metrics.  The  primary  goal 
was  to  provide  a  useful  instrument.  This  instrument  should 
help  an  organization's  management  to  better  understand  how 
their  metrics  measure  their  organization's  effectiveness  in 
terms  of  the  parameters  of  the  model . 

Research  Design 

In  keeping  with  the  purpose  of  this  research.  The 
project  was  designed  to  "collect  detailed  factual 
information  that  describes  existing  phenomena"  (1971:18). 

The  first  step  in  the  collection  process  was  to  develop  an 
instrument  based  on  the  Competing  Values  Model .  The  second 
was  to  validate  the  developed  instrument  by  testing  it  with 
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a  set  of  metrics  on  a  sample  of  people  able  to  judge  the 
utility  of  the  metrics.  First  I  will  give  an  overview  of 
the  instrument. 

Instrument-  The  Competing  Values  approach  to  analyzing 
an  organization's  effectiveness  was  used  as  a  starting 
point.  The  questions  were  then  designed  to  reflect  the  ends 
and  means  goals  of  each  of  the  quadrants  of  the  Competing 
Values  Model.  A  five-point  Likert  scale  was  used  to  elicit 
responses  from  subjects.  Each  of  the  24  questions  (six  per 
quadrant)  attempted  to  either  identify  an  end  or  a  mean 
within  one  of  the  four  quadrants.  Additionally,  standard 
semantic  differential  scales  (Osgood  and  Others,  1957)  were 
used  to  give  a  comparative  measure  of  peoples'  value 
judgments  about  the  metrics. 

The  five  Likert  response  points  were;  1  =  Of  slight 
use,  2  =  Of  some  use,  3  =  Of  moderate  use,  4  =  Of 
significant  use,  and  5  =  Extremely  useful. 

The  questions  associated  with  the  Human  Relations  (HR) 
quadrant  ends  and  means  were : 

Of  how  much  use  is  the  metric  in  determining. . . 

HR  Ends 

1 .  The  level  of  concern  members  have  for  the 

organization? 

2 .  The  level  of  commitment  members  feel  toward  the 

organization? 
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3.  The  level  of  morale  within  the  organization? 


HR  Means 

1.  The  degree  to  which  the  organization  uses 
discussion  among  members  to  set  goals  and/or  solve 
problems? 

2 .  The  degree  to  which  all  members  have  an  opportunity 
to  participate  in  organizational  decision  making? 

3 .  The  degree  of  openness  of  management  within  the 
organization? 

The  questions  associated  with  the  Open  Systems  (OS) 
quadrant  ends  and  means  were : 

Of  how  much  use  is  the  metric  in  determining. . . 

OS  Ends 

1.  The  organization's  ability  to  gain  support  from 
other  organizations'  to  expand  the  organization's 
capabilities? 

2.  The  organization's  ability  to  expand  its  resources 
to  grow? 

3.  The  organization's  ability  to  expand  or  increase 
maximum  output? 

OS  Means 

1.  The  level  of  innovation  within  the  organization? 
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2.  The  ability  of  the  organization  to  adapt  to  change? 

3 .  The  degree  of  insight  the  organization  has  of  its 
environment? 

The  questions  associated  with  the  Internal  Process  (IP) 
quadrant  ends  and  means  were : 

Of  how  much  use  is  the  metric  in  determining.  .  . 

IP  Ends 

1.  The  effectiveness  of  the  information  provided  by 
the  organization's  information  structure? 

2.  The  utility  of  the  organization's  rules, 
regulations,  policies,  or  procedures? 

3 .  The  quality  of  process  measurement  within  the 
organization? 

IP  Means 

1.  The  stability  of  operational  processes  within  the 
organization? 

2 .  The  level  of  control  by  management  over  processes 
within  the  organization? 

3.  The  ability  of  the  organization  to  sustain  a  smooth 
work  flow? 

The  questions  associated  with  the  Rational  Goal  (RG) 
quadrant  ends  and  means  were : 
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Of  how  much  use  is  the  metric  in  determining. . . 

R6  Ends 

1.  The  impact  the  organization's  products/services 
have  on  customers? 

2.  How  well  the  organization  uses  its  resources? 

3.  How  effective  the  organization  accomplishes  its 
goals? 

RG  Means 

1.  How  decisive  leaders  have  been  in  responding  to 
employees'  production-related  concerns? 

2.  How  clear  leaders  have  been  in  defining  operational 
goals? 

3.  How  directive  leaders  have  been  in  managing  the 
organization? 

Semantic  Differential  •  The  semantic  differential  was  used 
to  measure  perceived  meaning  of  a  word  or  concept  across 
three  major  parameters  (Activity,  Evaluative,  and  Potency) . 
Standard  items  for  each  scale  developed  by  Osgood,  Suci  and 
Tannebaum  were  used  (1957) .  The  standard  items  for  each 
parameter  are  in  displayed  Table  3 . 
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Table  3 .  Semantic  Differential  Parameters 


Activity 

Evaluative 

Potency 

Active  -  Passive 

Good  -  Bad 

Small  -  Large 

Slow  -  Fast 

Low  -  High 

Weak  -  Strong 

Dull  -  Sharp 

Bitter  -  Sweet 

Delicate  -  Rugged 

Calm  -  Excitable 

Worthless  -  Valuable 

Light  -  Heavy 

Initially,  the  dimensions  used  to  evaluate  a  concept  by 
subjects  must  be  determined.  The  complete  range  of  meaning 
for  a  concept  was  considered  the  semantic  space  of  a  concept 
(Osgood  and  others,  1957:31) .  The  relationships  among  the 
scores  on  the  three  scales  represents  a  measurement  of  the 
meaning.  In  this  research,  peoples'  semantic  space  for  each 
metric  was  being  sought  as  a  correlate. 

The  pairs  of  polar  opposites  are  combined  in  an 
instrument  which  uses  Likert -like  scaling.  This  gives  the 
subject  the  ability  to  agree  or  disagree  to  varying  degrees 
with  the  polar  extremes.  "The  purpose  of ...  factoring  work 
was  to  discover  the  'natural'  dimensionality  of  the  semantic 
space,,  the  system  of  factors  which  together  account  for  the 
variances  in  meaningful  judgments..."  (Osgood  and  others, 
1957:31).  "Both  the  Likert  and  semantic  differential 
formats  have  a  greater  rigor  and  structure  than  other 
question  formats.  These  formats  produce  data  suitable  to 
both  indexing  and  scaling"  (Babbie,  1986:376-77). 

The  order  of  the  scale  items  was  randomized  as  were  the 
positive  and  negative  ends  of  the  scale.  Isaac  and  Michael 
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point  to  the  importance  of  this  random  placement  in 
constructing  semantic  differentials  by  stating,  "arrange  the 
polar  adjective  pairs  so  that  the  favorable,  potent,  or 
active  end  of  the  scale  was  randomly  placed  in  a  right  or 
left  position  to  avoid  position  habits  in  the  response 
pattern"  (Isaac  and  Michael,  1981:146).  Although  I  do  not 
use  semantic  differentials  as  they  are  strictly  defined  by 
Isaac  and  Michael,  it  was  still  important  to  avoid  the 
"position  habits"  of  subject  responses  by  random  placement 
of  the  positively  connotated  word  in  the  instrument 
(1981:146) . 

Metrics  •  I  developed  a  hypothetical  metric  for  each 
quadrant  of  the  Competing  Values  Model ,  There  were  eight 
metrics  created  in  all,  one  each  for  means  and  ends.  These 
metrics  were  created  to  reflect  effectiveness  of  an  academic 
institution.  Each  metric  consisted  of  a  title  and  a  brief 
description  of  how  the  metric  could  be  calculated.  These 
metrics  will  be  used  to  determine  if  the  instrument  can 
predict  the  appropriate  quadrant  for  which  they  were 
created.  The  metrics  and  their  descriptions  follow: 

Metric  1  -  Faculty  Departure  Rate. 

It  is  calculated  by  using  the  percent  of  total  civilian 
faculty  who  leave  each  year  for  positions  with  other 
institutes . 
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Metric  2  -  Commander  Contact  Rate. 


It  is  calculated  by  the  number  of  meetings  per  year  by 
commandant /dean  with  students. 

Metric  3  -  Program  Manager  Effectiveness. 

It  is  calculated  by  the  percent  of  students  satisfied 
with  their  academic  program  management . 

Metric  4  -  Course  Criteria  Satisfaction. 

It  is  calculated  by  the  degree  to  which  the  instructor 
adhered  to  the  stated  course  grading  criteria. 

Metric  5  -  Program  Application  Rate. 

It  is  calculated  by  the  number  of  student  applications 
per  available  slot  for  each  program. 

Metric  6  -  Course  Innovation. 

It  is  calculated  by  the  percent  of  faculty  time  spent 
on  course  development/ improvement . 

Metric  7  -  Faculty  Publication  Rate. 

It  is  calculated  by  the  number  of  publications  per 
faculty  member  per  year. 

Metric  8  -  Thesis  Topics  Generated  by  Faculty. 

It  is  calculated  by  the  number  of  thesis  ideas 
suggested  by  faculty  as  percent  of  total  thesis  topics . 
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Selection  of  Subjects •  I  limited  the  selection  of  subjects 
to  faculty  and  students  within  the  Air  Force  Institute  of 
Technology.  I  assumed  this  would  give  the  subject  an 
adequate  base  of  experience  from  which  to  evaluate  the 
educational  metrics  used  in  this  research.  I  also  made  no 
attempt  to  make  this  a  multi -cultural  instrument.  This 
limits  the  subjects  to  English  speaking  "Western  or 
American"  members  of  society.  These  members  must  have  also 
used  metrics  within  an  organization. 

Sample  •  The  sample  consists  of  members  of  the  faculty 
and  student  body  of  the  Air  Force  Institute  of  Technology. 
This  sample  represents  college  graduates  with  a  minimum  of  a 
bachelors  degree.  All  are  English-speaking  although  for 
some  English  was  a  second  language.  Other  organizations' 
should  be  measured  in  the  future  to  establish  a  more 
comprehensive  data  base. 

Data  Collection  and  Analysis 

Each  person  recorded  their  responses  to  three  different 
metrics  directly  on  the  surveys.  Thirty  six  item  (24 
end/means  scale  and  12  semantic  differential)  responses  to 
each  metric  constituted  a  record. 

The  scale  scores  were  calculated  in  the  following 
manner.  The  competing  values  scale  scores  were  calculated 
as  the  mean  of  the  item  scores  for  each  scale.  The  semantic 
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differential  was  calculated  as  the  mean  of  items  associated 


with  each  semantic  differential  parameter. 

The  analysis  method  was  a  statistical  based  approach 
that  analyzed  the  relationship  of  the  metrics  to  the 
competing  values  scales  and  semantic  differential  scales. 

The  analysis  provided  information  on  mean  and  significant 
correlations . 

Methodological  Assumptions  and  Limitations 

There  are  limitations  to  this  methodology  in  spite  of 
efforts  to  use  highly  appropriate  tools.  The  first  was 
inherent  in  the  task.  This  instrument  deals  essentially 
with  meaning  and  interpretation.  Regardless  of  the  rigor  in 
the  instrument  and  its  administration,  scales  and  factors 
are  open  to  the  interpretation  of  each  reader  (Isaac  and 
Michael,  1971:103). 

My  sampling  method  was  subject  to  criticism  on  at  least 
three  fronts.  First,  the  entire  sample  was  voluntary.  And 
so,  there  was  no  way  to  know  if  the  responses  of  respondents 
(volunteers)  would  differ  significantly  from  non -respondents 
(non-volunteers) .  Second,  the  convenience  method  of 
sampling  drew  a  very  high  percentage  of  subjects  from 
military  backgrounds.  This  was  reported  in  the  findings  of 
the  next  chapter.  Third,  the  number  of  respondents  permits 
only  about  an  80-85%  level  of  significance  in  interpreting 
the  findings. 
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An  additional  threat  to  internal  validity  was  the 
maturation  process  (Campbell  and  Stanley,  1963:5).  Subjects 
could  conceivably  lose  interest,  tire,  or  learn  during  the 
course  of  completing  the  instrument.  I  countered  this 
threat  by  designing  the  instrument  to  be  as  short  as 
possible.  The  average  estimated  time  to  complete  it  was  15 
minutes . 

The  first  threat  to  external  validity  was  "interaction 
effects  of  selection  biases  and  the  experimental  variable" 
(Campbell  and  Stanley,  1963:6).  The  particular  nuance  of 
this  threat  that  was  applicable  here  concerns  the 
representativeness  of  the  sample.  The  predominance  of 
subjects  with  a  military  background  jeopardizes  any  attempt 
to  generalize  the  findings. 

The  last  threat  was  also  one  to  external  validity. 
Campbell  and  Stanley  call  it  the  threat  of  "reactive  effects 
of  experimental  arrangements"  (1963:6).  While  I  have  not 
introduced  a  treatment,  per  se,  to  the  subjects,  this  threat 
should  be  addressed.  There  was  a  danger  in  assuming  the 
instrument  developed  here  will  exhaustively  describe  the 
concept,  metric  evaluation.  Such  a  generalization  was  not 
appropriate .  I  have  attempted  to  construct  an  instrument 
that  would  test  the  parameters.  Subjects  are  limited  in  the 
ability  to  respond  by  the  scales  I  have  used  to  represent 
each  parameter.  If  there  are  additional  factors  to  metric 
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evaluation,  the  subjects  will  find  no  way  to  include  these 
in  the  responses  provided. 

Summary 

This  measure  should  help  an  organization  better 
understand  how  their  metrics  are  measuring  their 
organization.  It  might  help  them  identify  areas  that  may 
not  be  being  measured,  and  possibly  areas  where  they  are 
over  measuring.  Additionally,  it  might  indicate  if  they  are 
measuring  what  they  believe  they  are  measuring.  That  was, 
are  they  operating  the  way  they  claim  to  be. 

The  development  and  validation  of  the  survey  in  itself 
was  the  primary  objective  here.  The  testing  and  analysis  of 
the  measure  being  administered  to  one  organization  was  only 
the  first  step  in  validating  the  measure  a  useful  and 
valuable  instrument . 
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IV.  Data  Analysis 


Chapter  Overview 

This  chapter  presents  findings  of  the  research  effort 
as  described  in  Chapter  III,  Methodology.  The  findings  are 
presented  in  the  same  order  as  the  research  objectives. 

Research  Objective  One:  is  there  an  adequate  modei  available  on  which 
metric  evaluation  can  be  based? 

Research  objective  one  was  introduced  in  Chapter  II, 
Literature  Review.  As  expected,  there  was  an  adequate  model 
available  on  which  to  base  a  metric  evaluation  instrument. 

The  Competing  Values  Model  was  chosen  because  it  is 
more  encompassing  at  measuring  organization  effectiveness 
than  the  Goal  Attainment,  Systems,  or  Strategic 
Constituencies  approaches.  The  Competing  Values  Approach 
considered  thirty  indicators  of  organization  effectiveness 
within  its  framework.  Additionally,  the  framework  proposed 
by  the  Competing  Values  Model  lent  itself  to  be  used  as  a 
foundation  for  an  instrument  that  could  evaluate  metrics. 

It  is  appropriate  to  recap  the  Competing  Values  Model 
at  this  point.  It  is  summarized  in  Figure  3.  A  more 
complete  description  can  be  found  beginning  on  page  24  of 
Chapter  II.  The  vertical  axis  represents  decentralization 
on  one  end  and  centralization  at  the  other,  while  the 
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horizontal  axis  represents  people  on  one  end  and  the 
organization  on  the  other.  The  axis  between  the  horizontal 
and  vertical  axis  represents  means/process  on  one  end  and 
ends/goals  on  the  other.  This  framework  has  been  expanded 
(Figure  3)  to  form  a  more  constructive  framework  for 
illustrating  organization  effectiveness. 


Figure  3.  Competing  Values  Framework:  Effectiveness. 


The  vertical  and  horizontal  axes  cross  to  form  four 
quadrants.  Each  quadrant  represents  a  different  aspect  of 
organizational  effectiveness.  The  upper  left  quadrant 
identifies  with  the  humans  relations  model  and  emphasizes 
morale  and  human  resource  development.  The  upper  right 
quadrant  identifies  with  the  open  systems  model  and 
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emphasizes  flexibility,  readiness,  growth  and  external 
support.  The  lower  left  quadrant  identifies  with  the 
internal  process  model  and  emphasizes  information  management 
and  communication.  The  lower  right  quadrant  identifies  with 
the  rational  goal  model  and  emphasizes  planning,  goal 
setting  and  productivity. 

Each  model  has  a  polar  opposite  (Quinn,  1988:47) .  For 
instance,  the  open  systems  model,  which  emphasizes 
flexibility  and  external  support,  stands  opposite  to  the 
internal  process  model,  which  emphasizes  control  and 
internal  focus.  Additionally,  there  are  parallels  among  the 
models.  Human  relations  and  open  systems  model  both  place 
an  emphasis  on  flexibility.  There  are  other  similarities 
among  the  other  models.  Each  of  the  models  has  a  means  to 
achieve  an  end.  For  instance,  "in  the  internal  process 
model ...  information  management  and  communication  are  viewed 
as  a  means  of  arriving  at  stability  and  control"  (Quinn, 
1988:47).  This  model  provides  us  with  a  comprehensive 
understanding  and  an  integrated  framework  for  examining 
organization  effectiveness. 
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Research  Objective  Two:  Can  an  instrument  be  developed  to  evaluate 
metrics  against  such  a  model? 

Instrument-  it  is  appropriate  to  recap  the  scales  used 
in  the  instrument  at  this  time.  The  questions  were  designed 
to  fall  into  the  four  effectiveness  quadrants  of  the 
Competing  Values  Model .  Each  of  the  24  questions  attempted 
to  either  identify  an  end  or  a  mean  within  one  of  the  four 
quadrants.  Additionally,  standard  semantic  differential 
scales  were  used  to  give  a  comparative  measure  of  peoples' 
reactions  to  the  metrics. 

The  five  Likert  response  points  were;  1  =  Of  slight 
use,  2  =  Of  some  use,  3  =  Of  moderate  use,  4  =  Of 
significant  use,  and  5  =  Extremely  useful. 

The  questions  associated  with  the  Human  Relations  (HR) 
quadrant  ends  and  means  were : 

Of  how  much  use  is  the  metric  in  determining. . . 

HR  Ends 

1.  The  level  of  concern  members  have  for  the 

organization? 

2 .  The  level  of  commitment  members  feel  toward  the 

organization? 

3 .  The  level  of  morale  within  the  organization? 
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HR  Means 


1.  The  degree  to  which  the  organization  uses 
discussion  among  members  to  set  goals  and/or  solve 
problems? 

2 .  The  degree  to  which  all  members  have  an  opportunity 
to  participate  in  organizational  decision  making? 

3 .  The  degree  of  openness  of  management  within  the 
organization? 

The  questions  associated  with  the  Open  Systems  (OS) 
quadrant  ends  and  means  were : 


Of  how  much  use  is  the 
OS  Ends 

1.  The  organization's 
other  organizations  to 
capabilities? 

2.  The  organization's 
to  grow? 

3.  The  organization's 
maximum  output? 


metric  in  determining. . . 

ability  to  gain  support  from 
expand  the  organizations 

ability  to  expand  its  resources 

ability  to  expand  or  increase 


OS  Means 

1.  The  level  of  innovation  within  the  organization? 

2.  The  ability  of  the  organization  to  adapt  to  change? 
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3 .  The  degree  of  insight  the  organization  has  of  its 
environment? 

The  questions  associated  with  the  Internal  Process  (IP) 
quadrant  ends  and  means  were : 

Of  how  much  use  is  the  metric  in  determining. . . 

IP  Ends 

1 .  The  effectiveness  of  the  information  provided  by 
the  organizations  information  structure? 

2.  The  utility  of  the  organization's  rules, 
regulations,  policies,  or  procedures? 

3 .  The  quality  of  process  measurement  within  the 
organization? 

IP  Means 

1.  The  stability  of  operational  processes  within  the 
organization? 

2 .  The  level  of  control  by  management  over  processes 
within  the  organization? 

3 .  The  ability  of  the  organization  to  sustain  a  smooth 
work  flow? 

The  questions  associated  with  the  Rational  Goal  (RG) 
quadrant  ends  and  means  were : 
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Of  how  much  use  is  the  metric  in  determining. . . 

RG  Ends 

1.  The  impact  the  organization's  product s/services 
have  on  customers? 

2.  How  well  the  organization  uses  its  resources? 

3.  How  effective  the  organization  accomplishes  its 
goals? 


RG  Means 

1.  How  decisive  leaders  have  been  in  responding  to 
employees'  product ion- related  concerns? 

2.  How  clear  leaders  have  been  in  defining  operational 
goals? 

3 .  How  directive  leaders  have  been  in  managing  the 
organization? 

Response  Rates  •  Table  4  summarizes  the  response  rates 
from  the  group  we  administered  the  survey  to.  Of  the  105 
surveys  returned,  3  were  incomplete. 


Table  4.  Response  Rates 


Demographic  Data  •  As  in  Hicks  (1991)  ,  my  goal  was  to 
poll  subjects  of  college  age  or  older.  Since  all  of  the 
subjects  that  were  surveyed  were  either  professors  or  master 
degree  students  at  AFIT,  this  criteria  was  met. 

Reliability  Procedures-  I  measured  the  reliability  of  each 
ends/means  scale  as  constructed  to  reflect  the  eight  areas 
of  the  Competing  Values  Model.  I  used  Cronbach's  alpha. 
Cronbach's  alpha  calculation  produces  a  number  between  0  and 
1  which  reflects  the  internal  consistency  reliability  of  the 
items  in  a  scale.  The  higher  the  Cronbach's  alpha,  the  more 
reliable  the  scale.  Items  were  only  to  be  removed  from  a 
scale  if  their  removal  would  have  created  a  significant 
alpha  improvement.  In  this  manner,  I  hoped  to  improve  each 
area . 

Findings  For  Scales  •  I  will  present  tables  containing 
Cronbach's  alpha  for  each  set  of  questions  associated  with 
each  competing  values  scale  and  each  Semantic  Differential 
Scale.  The  first  set  of  statistics  will  contain  the 
Cronbach's  alpha  for  the  scale,  the  correlation  of  each 
question  with  the  total  and  the  Cronbach  alpha  for  each 
scale  if  the  separate  item  were  removed.  I  will  show  the 
final  alpha  and  the  scale  retained  to  describe  each  area. 

Scale  1  -  Human  Relations  Ends •  The  reliability  for 
the  area  in  Table  5  is  0.906823  which  indicates  this  is  a 
strong  scale  for  the  measurement  of  the  Human  Relations  Ends 


53 


Scale.  There  are  no  questions  which  could  be  deleted  to 
improve  the  reliability  of  the  scale. 


Table  5.  Cronbach  Alpha  -  Human  Relations  Ends  Scale 


1  Alpha  for  RAW  variables : 

0.906823 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

0.822404 

0.860399 

Question  20 

0.833668 

0.849565 

Question  21 

0.787582 

0.888993 

Scale  2  -  Human  Relations  Means  •  The  reliability  of 
the  area  in  Table  6  is  0.931779  which  indicates  this  is  a 
strong  scale  for  the  measurement  of  the  Human  Relations 
Means  Scale.  There  are  no  questions  which  could  be  deleted 
to  improve  the  reliability  of  the  scale. 


Table  6.  Cronbach  Alpha  -  Human  Relations  Means  Scale 


1  Alpha  for  RAW  variables:  0.931779  | 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

0.820571 

0.931373 

0.904662 

0.865199 

1  Question  24 

0.857677 

0.904039 

Scale  3  -  Internal  Process  Ends  •  The  reliability  of  the 
area  in  Table  7  is  0.792610  which  indicates  this  is  a  strong 
scale  for  the  measurement  of  the  Internal  process  Ends 
Scale.  While  deleting  question  15  would  improve  the 
reliability,  it  is  not  a  significant  improvement. 
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Table  7.  Cronbach  Alpha  -  Internal  Process  Ends  Scale 


1  Alpha  for  RAW  variables : 

0.792610 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

0.649221 

0.702968 

0.708555 

0.639439 

0.552988 

0.800786 

Scale  4  -  Internal  Process  Means  •  The  reliability  of  the 
area  in  Table  8  is  0.797796  which  indicates  this  is  a  strong 
scale  for  the  measurement  of  the  Internal  Process  Means 
Scale.  While  deleting  question  17  would  improve  the 
reliability,  it  is  not  a  significant  improvement. 


Table  8.  Cronbach  Alpha  -  Internal  Process  Means  Scale 


1  Alpha  for  RAW  variables : 

0.797796 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

0.759344 

0.594409 

0.530929 

0.833016 

1  Question  18 

0.648127 

0.718563 

Scale  5  -  Open  Systems  Ends  •  The  reliability  of  the 
area  in  Table  9  is  0.771575  which  indicates  this  is  a  strong 
scale  for  the  measurement  of  the  Open  Systems  Ends  Scale. 
There  are  no  questions  which  could  be  deleted  to  improve  the 
reliability  of  the  scale. 
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Table  9.  Cronbach  Alpha  -  Open  Systems  Ends  Scale 


1  Alpha  for  RAW  variables : 

0.771575 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

Question  4 

0.592362 

0.708491 

Question  5 

0.692111 

0.597053 

Question  6 

0.539515 

0.763661 

Scale  6  -  Open  Systems  Means •  The  reliability  of  the 
area  in  Table  10  is  0.861873  which  indicates  this  is  a 
strong  scale  for  the  measurement  of  the  Open  Systems  Means 
Scale.  There  are  no  questions  which  could  be  deleted  to 
improve  the  reliability  of  the  scale. 


Table  10.  Cronbach  Alpha  -  Open  Systems  Means  Scale 


1  Alpha  for  RAW  variables :  C 

). 861673 

Deleted 

Correlation 

Variable 

Alpha 

Question  1 

0.748349 

0.797383 

Question  2 

0.747397 

0.796958 

Question  3 

0.720512 

0.824402 

Scale  7  -  Rational  Goal  Ends  •  The  reliability  of  the 
area  in  Table  11  is  0.820508  which  indicates  this  is  a 
strong  scale  for  the  measurement  of  the  Rational  Goal  Ends 
Scale.  There  are  no  questions  which  could  be  deleted  to 
improve  the  reliability  of  the  scale. 
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Table  11.  Cronbach  Alpha  -  Rational  Goal  Ends  Scale 


1  Alpha  for  RAW  variables : 

0.820508 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

Question  7 

0.625506 

0.806649 

Question  8 

0.652728 

0.775183 

Question  9 

0.752936 

0.671806 

Scale  8  -  Rational  Goal  Means .  The  reliability  of  the 
area  in  Table  12  is  0.829987  which  indicates  this  is  a 
strong  scale  for  the  measurement  of  the  Rational  Goal  Means 
Scale .  There  are  no  questions  which  could  be  deleted  to 
improve  the  reliability  of  the  scale. 


Table  12.  Cronbach  Alpha  -  Rational  Goal  Means  Scale 


Alpha  for 

RAW  variables : 

0.829987 

Deleted 

Correlation 

Variable 

Alpha 

Question  10 

0.633778 

0.820733 

0.668255 

0.784569 

0.769077 

0.685726 

Scale  Means  by  Metric  •  Table  13  shows  the  research 
metrics  across  the  top  and  the  designed  scales  down  the 
side.  Looking  down  each  metrics  column,  two  of  the  scales, 
HRE  and  HRM,  means  are  the  highest  for  the  metric  that  it 
purports  to  measure,  while  one  other,  OSM,  mean  is  the 
highest  for  OSE,  which  is  in  the  same  quadrant. 

The  probability  that  one  of  the  metrics  would  randomly 
fall  in  the  right  scale  is  .125.  So,  the  probability  that 
two  would  fall  on  the  right  scale  is  .015625.  Additionally, 
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the  probability  that  a  metric  would  randomly  fall  in  the 
correct  quadrant  is  .25.  So,  the  probability  that  three  of 
the  metrics  would  fall  in  the  correct  quadrant,  as 
predicted,  is  .015625. 


Table  13.  Means  for  Competing  Values  Scales  by  Metric 


HRE 

Metric 

HRM 

Metric 

IPE 

Metric 

I  PM 

Metric 

OSE 

Metric 

OSM 

Metric 

RGE 

Metric 

RGM 

Metric 

N  =  14 

N  =  13 

N  =  13 

N  =  13 

N  =  13 

N  =  13 

N  =  13 

N  =  13 

HRE 

2.6923 

3.2821 

2.3846 

2.2821 

2.3333 

2.3590 

2.3077 

HRM 

2.5476 

3;.;:;3077 

2.5897 

1.8974 

1.8205 

1.5385 

1.5897 

2.0256 

IPE 

2.0238 

1.8462 

2.4872 

2.4103 

2.3077 

2.0256 

1.7949 

1.8462 

I  PM 

2 . 5714 

1.7436 

2.4103 

2.2308 

2.1026 

1.7692 

1.9744 

1.7436 

OSE 

2.9048 

2.5897 

3.0000 

1.9231 

2.4103 

2.7436 

2.6154 

2.8205 

OSM 

2.5714 

1.9487 

2.6667 

1.7692 

2.6923 

2.3590 

2.2308 

RGE 

2.5238 

2.1539 

3.3846 

|||||;;|||| 

2.7949 

2.4615 

2.4615 

2 . 0513 

RGM 

2.8333 

2.5897 

2.8974 

2.1282 

2.3077 

2.1795 

1.7436 

1.6923 

Table  14  was  derived  by  adding  the  two  scale  means 
within  a  particular  quadrant  and  dividing  it  by  2.  The 
table  indicates  that  four  of  the  metrics,  HRE,  HRM,  OSE,  and 
OSM,  fall  into  the  predicted  quadrants.  The  probability  of 
this  happening  randomly  is  0.003906. 


Table  14.  Means  for  Competing  Values  Quadrants  by  Metric 


■ 

HRE 

Metric 

HRM 

Metric 

IPE 

Metric 

I  PM 

Metric 

OSE 

Metric 

OSM 

Metric 

RGE 

Metric 

RGM 

Metric 

HR 

3.17855 

3  ■ 

2.9359 

2.141 

2.0513 

1.9359 

1.97435 

2.16665 

IP 

2.2976 

1.7949 

2.44875 

2.32055 

2.20515 

1.8974 

1.88465 

1.7949 

OS 

2.7381 

2.2692 

2.83335 

1.84615 

2.5513 

2.5513 

2.65385 

2.52565 

RG 

2.67855 

2.3718 

3.141 

2.4359 

2.5513 

2.3205 

2.10255 

1.8718 
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Alpha  for  Activity  Semantic  Differential  •  The  reliability  of 
the  area  in  Table  15  is  0.811975  which  indicates  this  is  a 


strong  scale  for  the  measurement  of  the  Activity  Semantic 
Differential  Scale  and  validates  Osgood,  and  others  (1957) 
delineation  of  the  scale  items  for  this  research. 


Table  15.  Cronbach  Alpha  -  Activity  Semantic  Differential  Scale 


1  Alpha  for  RAW  variables :  ^ 

0.811975 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

Question  1 

0.670136 

0.774579 

Question  2 

0.713020 

0.724934 

Question  3 

0.688402 

0.741529 

Question  8 

0.554325 

0.806900 

Alpha  for  Evaluative  Semantic  Differential  •  The  reliability 
of  the  area  in  Table  16  is  0.834337  which  indicates  this  is 
a  strong  scale  for  the  measurement  of  the  Evaluative 
Semantic  Differential  Scale  and  validates  Osgood  and  others 
(1957)  delineation  of  the  scale  items  for  this  research. 
While  deleting  question  10  would  improve  the  reliability  of 
the  scale,  it  is  not  a  significant  improvement. 
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Table  16.  Cronbach  Alpha  -  Evaluative  Semantic  Differential  Scale 


Alpha  for 

RAW  variables : 

0.834337 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

Question  4 

0.799507 

0.724091 

Question  6 

0.628784 

0.805927 

Question  10 

0.531604 

0.857238 

Question  12 

0.804600 

0.726893 

Alpha  for  Potency  Semantic  Differential  •  The  reliability 
of  the  area  in  Table  17  is  0.844066  which  indicates  this  is 
a  strong  scale  for  the  measurement  of  the  Potency  Semantic 
Differential  Scale  and  validates  Osgood  and  others  (1957) 
delineation  of  the  scale  items  for  this  research. 


Table  17.  Cronbach  Alpha  -  Potency  Semantic  Differential  Scale 


1  Alpha  for  Raw  variables : 

0.844066 

Deleted 

Correlation 

Variable 

with  Total 

Alpha 

Question  5 

0.752187 

0.770428 

Question  7 

0.609659 

0.838247 

0.741175 

0.775781 

0.680289 

0.817875 

Means  for  Semantic  Differential  Scales  by  Metric  Table  18 
informs  us  of  the  value  judgments  of  the  subjects  for  the 
scales  in  terms  of  activity,  evaluative,  and  potency 
Semantic  differentials.  The  Internal  Process  Ends  metric 
has  the  highest  mean  for  all  three  Semantic  Differential 
Scales,  while  the  Rational  Goal  metrics  rank  second  and 
third  overall.  All  of  the  metrics  except  HRE  had  their 
highest  score  on  the  Evaluative  Semantic  Differential  Scale. 
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Table  18.  Means  for  Semantic  Differential  Scales  by  Metric 


Var 

HRE 

Metric 

HRM 

Metric 

I  PE 

Metric 

I  PM 

Metric 

OSE 

Metric 

OSM 

Metric 

RGE 

Metric 

RGM 

Metric 

N  =  13 

N  =  13 

N  =  13 

N  =  12 

N  =  12 

N  =  13 

N  =  13 

N  =  13 

Act 

iiliiiiiil 

4.1731 

4.6154 

3.3958 

3.5833 

3.9808 

3.9615 

|||;;||;|||: 

Eval 

3.6923 

4.8077 

3.7917 

3.9792 

4.1154 

4.4231 

Pot 

4.0192 

3.7500 

4.3462 

3.4583 

3.7500 

3.9423 

4.3077 

4.0385 

Correlations  •  In  Table  19,  we  find  that  all  of  the 
scales  are  highly  correlated  with  the  Rational  Goal  Scales. 


Table  19.  Correlation  for  Competing  Values  Scales  by  Competing  Values 

Scales 


HRE 

HRM 

I  PE 

I  PM 

OSE 

OSM 

RGE 

HRM 

0.67 

I  PE 

0.59 

0.54 

I  PM 

0.61 

0.53 

0.71 

OSE 

0.55 

0.50 

0.41 

0.46 

OSM 

0.45 

0.35 

0.50 

0.59 

0.61 

RGE 

0.65 

0.48 

0.71 

0.72 

0.61 

0.65 

RGM 

0.75 

0.73 

0.68 

0.69 

0.57 

0.55 

0.71 

p  <  .0001  significance 


Table  20  demonstrates  the  OSE  scale  has  the  highest 
overall  average  correlation  with  the  Semantic  Differential 
Scales.  On  average  the  Competing  Values  Scales  correlate 
higher  with  the  Evaluative  Semantic  Differential  Scale. 
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Table  20.  Correlation  Competing  Values  Scales  by  Semantic  Differential 

Scales 


HRE 

HRM 

IPE 

I  PM 

OSE 

OSM 

RGE 

ROM 

Act 

0.3927 

0.3006 

0.1994 

0.3717 

0.3306 

0.2955 

Eval 

0.3530 

0.3131 

0.2750 

0.3770 

0.3114 

Pot 

0.3884 

0.2224 

0.3045 

0.2099 

0.3298 

0.3764 

0.2945 

0 . 2497 

p  <  .05  significance 


In  view  of  the  information  in  table  20,  two  regression 
analyses  were  performed  to  determine  which  scales  and  scale 
quadrants  were  most  predictive  of  the  ratings  on  the 
semantic  differential  scales.  A  composite  semantic 
differential  score  based  on  the  mean  of  all  12  items  was 
used  as  the  dependent  variable.  As  can  be  seen  in  Table  21 
only  the  OSE  scale  attains  significance  at  the  <  .01. 
However,  the  HRE,  OSM,  and  IPE  scales  are  approaching  the 
.05  level. 
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Table  21.  Regression  for  Semantic  Differentials  by  Scale 


I  Analysis  of  Variance  | 

Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Prob>F 

Model 

8 

39.90362 

4.98795 

5.530 

0.0001 

Error 

93 

83.89166 

0.90206 

C  Total 

101 

123.79528 

Root  MSE 

0.94977 

R- square 

0.3223 

Dep  Mean 

4.07761 

Adj  R-sq 

0.2640 

C.V. 

23.29227 

Parameter  Estimates 

Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=0 

Prob  >  1 T 1 

INTERCEP 

1 

2.419851 

0.29530983 

8.194 

0.0001 

HRE 

1 

0.236090 

0.12237136 

1.929 

0.0567 

HRM 

1 

0.076467 

0.12555843 

0.609 

0.5440 

IPE 

1 

0.267462 

0.15222105 

1.757 

0.0822 

I  PM 

1 

-0.244283 

0.17456968 

-1.399 

0.1650 

OSE 

1 

0.353919 

0.13402862 

2.641 

0.0097 

OSM 

1 

0.231302 

0.12932470 

1.789 

0.0769 

RGE 

1 

-0.043945 

0.15359483 

-0.286 

0.7754 

RGM 

1 

-0.247668 

0.16838742 

-1.471 

0.1447 

Table  22  shows  the  combined  scales  in  the  OS  quadrant 
attain  significance  at  the  .002  level  and  the  scales  in  the 
HR  quadrant  are  at  the  .0437  level  of  significance. 
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Table  22.  Regression  for  Semantic  Differentials  by  Quadrants. 


1  Analysis  of  Variance  | 

Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Prob>F 

Model 

4 

34.64343 

8.66086 

9.423 

0.0001 

Error 

97 

89.15184 

0.91909 

C  Total 

101 

123.79528 

Root  MSE 

0.95869 

R- square 

0.2798 

Dep  Mean 

4.07761 

Adj  R~sq 

0.2501 

C.V. 

23.51111 

Parameter  Estimates 

Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=:0 

Prob  >  1 T 1 

INTERCEP 

1 

2.445134 

0.29345307 

8.332 

0.0001 

HR 

1 

0.276335 

0.13523810 

2.043 

0.0437 

IP 

1 

0.069017 

0.19003477 

0.363 

0.7173 

OS 

1 

0.583621 

0.14813481 

3 . 940 

0.0002 

RG 

1 

-0.265442 

0.20862964 

-1.272 

0.2063 

Summary 

This  chapter  began  with  the  information  concerning  the 
administration  of  the  initial  instrument.  Recall  that  the 
sample  size  used  for  data  analysis  was  105.  The  demographic 
statistics  show  the  sample  to  be  college  educated. 

The  bulk  of  this  chapter  was  spent  relating  information 
concerning  the  eight  areas.  I  presented  Cronbach's 
coefficient  of  reliability  alpha  for  each  of  the  areas. 

These  alphas  ranged  from  a  low  of  77  percent  to  a  high  of  93 
percent.  Five  reliability  coefficients  exceeded  0.800  and 
eight  exceeded  0.700.  Next,  I  examined  the  correlations  of 
the  scales  and  metrics  among  each  other. 
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Conclusions  and  recommendations  based  on  these  findings 


are  included  in  Chapter  V. 
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V.  Conclusion 


Chapter  Overview 

In  this  chapter  are  the  conclusions  reached  during  the 
research.  The  conclusions  presented  will  follow  the  order 
of  the  preceding  chapters.  I  will  first  address  conclusions 
derived  from  the  literature  review,  followed  by  conclusions 
from  the  methodology.  Finally,  I  will  present  conclusions 
related  to  the  data  I  obtained. 

Furthermore  in  this  chapter  I  will  outline  some 
recommendations  for  future  research  concerning  the  scales 
that  were  developed.  Also,  I  will  present  recommendations 
concerning  future  uses  of  like  instruments  developed  during 
my  research. 

Lastly,  I  will  discuss  my  successes  or  deficiencies  in 
obtaining  my  research  objectives.  I  address  each  objective 
and  provide  my  reasoning  concerning  my  research  completion. 

Conclusions 

Conclusions  From  the  Literature  Review.  After  examining  four 
popular  approaches  to  measuring  organization  effectiveness, 

I  selected  the  approach  to  model  my  instrument  after.  I 
chose  the  Competing-Values  Model  because  it  encompassed  far 
more  aspects  of  organization  effectiveness  than  the  other 
approaches  did. 
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Conclusions  From  the  Methodology-  The  decision  to  use  the  5 
point  Likert  scale  in  the  development  of  the  instrument 
proved  to  be  valuable.  The  scale  provided  the  user  with  an 
easy  to  use  scale  and  it  provided  me  with  precise  data. 

Conclusions  From  the  Data »  The  instrument  type  can  be 
completed  without  difficulty  by  most  subjects.  Less  than  10 
percent  of  the  subjects  returned  the  instrument  without 
successfully  completing  it. 

All  of  the  scales  had  a  Cronbach's  Alpha  of  .70  or 
higher.  This  indicates  that  the  scales  are  useful.  What  is 
important  now,  is  determining  how  they  can  best  be  used. 

The  HRM  and  HRE  metric's  correlated  highest  with  the 
HRM  and  HRE  scales,  respectively.  This  indicates  that  the 
metric's  I  developed  may  in  fact  be  true  HRM  and  HRE 
metric's,  and  people  are  able  to  recognize  them  as  an  HRM  or 
HRE  metric . 

The  OSM  metric  I  developed  correlated  highest  with  the 
OSE  scale.  This  is  at  least  in  the  correct  quadrant.  This 
may  indicate  that  people  are  unable  to  distinguish  between 
the  subtle  differences  in  OSE  and  OSM.  This  may  also 
indicate  that  the  metric  was  incorrectly  conceptualized  upon 
creation. 

The  IPE  metric  correlated  highly  with  several  of  the 
scales.  This  may  indicate  that  the  metric  could  be 
comprised  of  several  aspects  of  the  different  quadrants. 

One  of  the  assumptions  was  that  each  metric  would  only  be 
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identified  in  one  quadrant.  However,  the  data  may  indicate 
that  several  of  the  metrics  are  overlapping  into  one  or  more 
of  the  quadrants.  This  could  possibly  mean  that  managers 
may  design  metrics  for  a  specific  quadrant  but  actually  get 
the  benefit  of  measurement  in  more  than  one  quadrant.  This 
could  be  compared  to  shooting  a  shotgun  at  a  bird  in  a 
flock,  and  getting  several  birds  instead  of  the  one  you  were 
aiming  at . 

The  research  shows  that  although  the  RG  scales  are  not 
significant  in  any  regression  on  the  Semantic  Differentials, 
they  do  highly  correlate  with  all  the  other  metric 
measurement  scales.  This  might  mean  that  by  setting 
meaningful  goals  in  the  OS,  HR,  and  IP  quadrants,  managers 
may  derive  gains  in  the  RG  quadrant.  In  other  words,  it 
might  mean  that  it  is  more  beneficial  to  achievement  of  the 
goals  in  the  RG  quadrant  by  setting  better  goals  in  the 
other  three . 

The  Semantic  Differentials  all  had  a  Cronbach's  Alpha 
of  0.80  or  better.  This  also  indicates  that  the  scales  are 
useful.  All,  except  HRE,  of  the  metric's  correlated  the 
highest  with  the  Evaluative  Semantic  Differential  scale. 
Additionally,  5  out  of  the  8  scales  correlated  highest  with 
the  Evaluative  Semantic  Differential  scale.  This  may 
indicate  that  the  scales  and  metrics  used  were  viewed  as 
being  good  evaluators  of  what  they  were  purport  to  measure . 
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Recommendations 


Recommendations  Concerning  the  Instrument-  The  instrument  in 
Appendix  A  is  designed  to  test  specific  scales.  My  first 
recommendation  is  that  the  instrument  should  be  retested  to 
verify  the  alpha's.  This  retest  should  be  administered  to  a 
larger  sample  size.  The  sample  size  would  ideally  allow 
conclusions  to  be  drawn  at  the  0.01  level  of  significance. 

If  low  coefficients  of  reliability  are  discovered  for 
any  of  the  scales,  I  recommend  a  reconstruction  of  those 
sets  of  items .  Then  the  improved  scales  should  be 
substituted  into  the  existing  instrument. 

The  metrics  could  have  been  submitted  to  a  wider  panel 
of  experts  for  evaluation  before  the  distribution  of  the 
survey  was  accomplished.  In  that  way  I  might  have  been  able 
to  develop  more  accurately  targeted  metrics  and  the 
"splashover"  effect  into  several  quadrants  might  have  been 
minimized. 

Recommendations  Concerning  Methodology  •  The  instruments  can 
be  used  in  at  least  three  different  ways.  First,  the 
instrument  can  be  used  to  determine  which  quadrant  or 
quadrants  a  particular  metric  may  lie.  Second,  the 
instrument  might  be  used  to  validate  where  other  instruments 
have  identified  a  particular  metric  to  lie.  Finally,  it  can 
be  used  with  an  organization's  entire  "constellation"  of 
metrics  to  determine  if  managers  are  targeting  the  areas  of 
importance  for  an  organization. 
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Recommendations  For  Future  Research-  Future  research  areas 
might  include  further  development  of  the  instrument  to 
improve  the  effectiveness  of  the  instrument  at  identifying 
the  quadrant (s)  for  a  particular  metric.  This  might  be 
better  accomplished,  if  the  metrics  being  evaluated  can  be 
pinpointed  to  one  and  only  one  quadrant.  Additionally,  the 
regression  analysis  presented  at  the  end  of  Chapter  IV 
provides  some  tantalizing  possibilities  in  terms  of  goal¬ 
setting.  Areas  of  research  might  address  the  question  of 
whether  or  not  organizational  members  respond  more 
positively  to  goals  represented  by  various  scales  (OSE,  IPE, 
HRE)  which  elicit  positive  ratings  on  the  Semantic 
Differential  Scales. 

Revisiting  the  Research  Objectives 

I  was  successful,  at  least  on  a  limited  basis,  in 
identifying  a  model  of  organization  effectiveness  in  which 
an  instrument  can  be  developed  to  evaluate  metrics.  The 
Competing  Values  Model  at  this  point  in  time  appears  to  be 
the  most  encompassing  model  of  organization  effectiveness 
and  the  model  of  choice . 

I  developed  an  instrument  to  evaluate  organization 
effectiveness  against  the  Competing  Values  Model.  As  to  how 
effective  the  instrument  is  or  how  useful  it  might  become, 
is  left  for  further  researchers  to  determine. 
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However,  I  may  have  conceptualized  the  problem 
incorrectly.  I  perceived  that  a  particular  metric  would 
fall  within  one  and  only  one  quadrant  of  the  competing 
values  model.  The  data  collected  from  this  research  may 
indicate  otherwise. 

Summary 

Each  of  the  research  objectives  established  for  this 
thesis  were  met.  The  results  provide  a  rich  basis  for 
future  research  in  instrument  development  for  evaluation  of 
metrics  that  measure  organization  effectiveness. 
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Appendix  A.  Metric  Evaluation  Instrument 

Metric  Evaluation  Instrument  [*V.l] 


Metrics  are  used  to  tell  us  about  characteristics  of  the  processes  within  our  organizations. 
This  instrument  was  developed  to  indicate  to  us  how  well  a  specific  metric  is  accomplishing  that 
goal. 


There  are  three  sections  in  this  evaluation.  The  first  section  provides  you  with  the 
description  of  a  metric  used  within  your  organization.  The  second  section  asks  you  to  analyze 
the  metric  as  to  how  important  it  is  in  terms  of  evaluating  certain  processes  and  outcomes  related 
to  organizational  effectiveness.  The  third  section  asks  you  to  relate  the  metric  in  terms  of  some 
word  pairs.  There  are  no  right  or  wrong  answers.  We  are  simply  trying  to  determine  what  the 
metric(s)  mean(s)  to  you. 


Section  1  -  Metric  Description 

The  metric  you  are  going  to  evaluate  is  called 
It  is  calculated  by  . 


Section  2  -  Process  and  Outcome  Evaluations 
In  responding  to  the  following  group  of  items  you  may  use: 

a.  The  information  provided  about  the  metric  in  Section  1 

b.  What  you  know  about  how  managers  use  the  metric  to  evaluate  your  organization 

Your  task  is  to  decide  how  useful  the  metric  is  in  evaluating  each  of  the  processes  or 
outcomes  described.  Circle  your  response  on  the  scale  to  the  right  of  each  item.  Use  the 
following  responses  to  make  your  judgments: 

1  2  3  4  5 

Of  slight  Of  some  Of  moderate  Of  significant  Extremely 

use  use  use  use  useful 


PLEASE  GO  ON  TO  THE  NEXT  PAGE 


72 


The  metric  you  are  going  to  evaluate  is  called  . 

It  is  calculated  by  . 

1  2  3  4  5 

Of  slight  Of  some  Of  moderate  Of  significant  Extremely 
use  use  use  use  useful 

Of  how  much  use  is  the  metric  in  determining... 

I .  ...the  organization’s  ability  to  gain  support  from  other  organizations  to 


expand  the  organizations  capabilities?  1  2 

2.  ...the  organization’s  ability  to  expand  its  resources  to  grow?  1  2 

3.  ...the  organization’s  ability  to  expand  or  increase  maximum  output?  1  2 

4.  ...the  level  of  iimovation  within  the  organization?  1  2 

5.  ...theability  of  the  organization  to  adapt  to  change?  1  2 

6.  ...the  degree  of  insight  the  organization  has  of  its  environment?  1  2 

7.  ...the  impact  the  organization’s  products/services  have  on  customers?  1  2 

8.  ...how  well  the  organization  uses  its  resources?  1  2 

9.  ...how  effective  the  organization  accomplishes  its  goals?  1  2 

10.  ...how  decisive  leaders  have  been  in  responding  to  employees’ 

production-related  concerns?  1  2 

1 1 .  ...how  clear  leaders  have  been  in  defining  operational  goals?  1  2 

12.  ...how  directive  leaders  have  been  in  managing  the  organization?  1  2 

13.  ...the  effectiveness  of  the  information  provided  by  the  organization’s 

information  structure?  1  2 

14.  ...the  utility  of  the  organization’s  rules,  regulations,  policies,  or  procedures?  1  2 

1 5 ...  .the  quality  of  process  measurement  within  the  organization?  1  2 


PLEASE  GO  ON  TO  THE  NEXT  PAGE 


3  4  5 
3  4  5 
3  4  5 
3  4  5 
3  4  5 
3  4  5 
3  4  5 
3  4  5 
3  4  5 

3  4  5 
3  4  5 
3  4  5 

3  4  5 
3  4  5 
3  4  5 
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The  metric  you  are  going  to  evaluate  is  called  . 

It  is  calculated  by  . 

1  2  3  4  5 

Of  slight  Of  some  Of  moderate  Of  significant  Extremely 

use  use  use  use  useful 


Of  how  much  use  is  the  metric  iu  determiuiug... 

16.  ...the  stability  of  operational  processes  within  the  organization?  1 

17.  ...the  level  of  control  by  management  over  processes  within  the  organization?  1 

1 8.  ...the  ability  of  the  organization  to  sustain  a  smooth  work  flow?  1 

19.  ...the  level  of  concern  members  have  for  the  organization?  1 

20.  ...the  level  of  commitment  members  feel  toward  the  organization?  1 

21 .  ...the  level  of  morale  within  the  organization?  1 

22.  ...the  degree  to  which  the  organization  uses  discussion  among  members 

to  set  goals  and/or  solve  problems?  1 

23.  ...the  degree  to  which  all  members  have  an  opportunity  to  participate  in 

organizational  decision  making?  1 

24.  ...the  degree  of  openness  of  management  within  the  organization?  1 


2  3  4  5 
2  3  4  5 
2  3  4  5 
2  3  4  5 
2  3  4  5 
2  3  4  5 

2  3  4  5 

2  3  4  5 
2  3  4  5 


PLEASE  GO  ON  TO  THE  NEXT  PAGE 
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Section  3  -  Metric  Word  Relations 


In  this  section  you  will  be  presented  with  sets  of  word  pairs.  Your  task  is  to  decide 
which  word  you  feel  most  accurately  describes  the  metric  and  place  an  “X”  in  the  space  nearer 
that  word.  The  more  strongly  you  feel  a  word  describes  the  metric  the  closer  to  it  you  would  put 
the  “X.”  If  you  felt  that  neither  word  described  the  metric,  or  that  they  both  described  it  equally 


well,  you  would  place  your  “X” 

on  the  center  blank. 

L_ 

1 

active _ 

:  :  :  :  : 

:  :  passive 

dull _ 

. 

:  :  sharp 

excitable _ 

. 

:  :  calm 

good _ 

:  :  bad 

heavy _ 

_•  - •  - •  - *  - *  _ 

:  :  light 

high _ 

. 

:  :  low 

rugged _ 

:  :  delicate 

slow _ 

. 

:  :  fast 

small _ 

_*  - •  , — •  - •  - •  _ 

:  :  large 

sweet _ 

. 

:  :  bitter 

weak _ 

_•  - • 

:  :  strong 

worthless 

:  :  :  :  : 

:  :  valuable 
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Previous  research  has  resulted  in  the  development  of 
instruments  to  measure  organizational  effectiveness.  This  thesis 
attempts  to  develop  an  instrument  to  measure  how  effective  the 
organization's  metrics  are  in  relation  to  their  organization 
effectiveness  model,  specifically,  to  the  Competing  Values  Model  of 
organization  effectiveness  developed  by  Quinn  and  Rohrbaugh.  This 
should  aid  managers  in  determining  whether  or  not  they  are 
measuring  what  they  think  they  are  measuring  and  how  effective  they 
are  at  measuring  organizational  effectiveness.  The  instrument 
developed  can  help  determine  the  specific  aspects  of  organizational 
effectiveness  which  metrics  are  measuring  according  to  the 
Competing  Values  Model. 
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